-- Logs begin at Tue 2022-05-24 09:17:13 UTC, end at Tue 2022-05-24 11:05:25 UTC. -- May 24 09:51:45.000755 dv-cinder-843118-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 24 09:51:45.009791 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue May 24 09:51:45 2022] *** May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: nodename: dv-cinder-843118-2-iscsi May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: machine: x86_64 May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: clock source: unix May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: pcre jit disabled May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: detected number of CPU cores: 4 May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: current working directory: / May 24 09:51:45.011801 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: detected binary path: /usr/bin/uwsgi-core May 24 09:51:45.012345 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: your processes number limit is 47680 May 24 09:51:45.012345 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: your memory page size is 4096 bytes May 24 09:51:45.012345 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: detected max file descriptor number: 2048 May 24 09:51:45.012345 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: lock engine: pthread robust mutexes May 24 09:51:45.012345 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: thunder lock: enabled May 24 09:51:45.012345 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 24 09:51:45.012345 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 24 09:51:45.059027 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: Python main interpreter initialized at 0x55d42a5a29c0 May 24 09:51:45.059027 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: python threads support enabled May 24 09:51:45.059027 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: your server socket listen backlog is limited to 100 connections May 24 09:51:45.059027 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: your mercy for graceful operations on workers is 90 seconds May 24 09:51:45.059387 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: mapped 403077 bytes (393 KB) for 2 cores May 24 09:51:45.059462 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: *** Operational MODE: preforking *** May 24 09:51:45.059731 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: *** uWSGI is running in multiple interpreter mode *** May 24 09:51:45.059731 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: spawned uWSGI master process (pid: 80392) May 24 09:51:45.059932 dv-cinder-843118-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 24 09:51:45.060876 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: spawned uWSGI worker 1 (pid: 80404, cores: 1) May 24 09:51:45.060876 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: spawned uWSGI worker 2 (pid: 80405, cores: 1) May 24 09:51:45.062322 dv-cinder-843118-2-iscsi devstack@keystone.service[80392]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 24 09:51:50.765820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 09:51:50.765820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: from cryptography.utils import int_from_bytes May 24 09:51:50.766609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 09:51:50.766609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: from cryptography.utils import int_from_bytes May 24 09:51:50.782224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 09:51:50.782224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: from cryptography.utils import int_from_bytes May 24 09:51:50.782707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 24 09:51:50.782707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: from cryptography.utils import int_from_bytes May 24 09:51:50.816190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 09:51:50.816374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 09:51:50.816564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] command line args: [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 09:51:50.816830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 09:51:50.817005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ================================================================================ {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 09:51:50.817254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] admin_token = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.817517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] config_dir = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.817703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.817883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] config_source = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.818061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] debug = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.818257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.818625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.818801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.819053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.819300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.819528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] list_limit = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.819705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] log_config_append = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.819879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.820053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] log_dir = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.820226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] log_file = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.820456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.820712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.820941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.821182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.821390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.821565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.821805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.822047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.822276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.822506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.822735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.822962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.823204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.823445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.823717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.824198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.35/identity {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.824377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] publish_errors = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.824606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.824833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.825008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.825254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.825508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.825741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] standard_threads = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.825970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.826209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.826383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.826556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] use_journal = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.826729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] use_json = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.826900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] use_stderr = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.827072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] use_syslog = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.827298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.827549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.827785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.828039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.828272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.828515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.828747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.828989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.829221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.829486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.829730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.829975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.830200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 24 09:51:50.830284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.830403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 24 09:51:50.830490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.830596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] command line args: [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 24 09:51:50.831383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.831504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 24 09:51:50.831610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.831723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ================================================================================ {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 24 09:51:50.831930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.832021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] admin_token = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.832093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.832168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] config_dir = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.832238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.832305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.832401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] config_source = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.832489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.832565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] debug = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.832703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.832796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.832968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.833042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.833171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.833247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.833449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.833526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.833697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.833774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.833925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] list_limit = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.834006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.834104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] log_config_append = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.834186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.834264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.834370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.834447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] log_dir = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.834596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.834673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] log_file = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.834828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.834912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.835071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.835147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.835314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.835389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.835538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.835619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.835721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.835797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.835895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.835993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.836136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.836237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.836391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.836468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.836626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.836707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.836858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.836935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.837090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.837166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.837343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.837423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.837590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.837668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.837815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.837890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.838043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.838125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.838271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.838488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.838607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.35/identity {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.838732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.838824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] publish_errors = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.838958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.839034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.839190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.839266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.839367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.839450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.839613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.839690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.839847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.839921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.840080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] standard_threads = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.840156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.840303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.840377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.840535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.840617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.840695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.840764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.840873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] use_journal = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.840985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.841066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] use_json = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.841212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] use_stderr = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.841320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.841399 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] use_syslog = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.841469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.841630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 24 09:51:50.841707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.841885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.841972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.842945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.843992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844224 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.844933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845168 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.845967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.846123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.846203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.846384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.846620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.846701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.846873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.846973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.847935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.848938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.849960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.850939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851273 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.851969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] auth.external = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.852886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] auth.password = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853521 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] auth.token = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.853987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.854993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] credential.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.855929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.856901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.857819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.858963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859271 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.859903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.860915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.861877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.862938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.863991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.864907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.865103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.865189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.865373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.865464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.865611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.865688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.865866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.866011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.866126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.866304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.866382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.866583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] auth.external = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.866659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.866814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] auth.password = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] auth.token = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.867851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.868962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] credential.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.869953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.870961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.871890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.872836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.873964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.874968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.875925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.876860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.877012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.877089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.877436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.878994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.879947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.880888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.881854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.882936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.883941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.884976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.885950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] role.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.886889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] role.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.887833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.888004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.888083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.888242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.888323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.888477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.888819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.888978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.889945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.890907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.891861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.892839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.893008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.893085 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.893938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.894958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.895909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.896998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.897968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.898981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.899913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.connection = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.900877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] role.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] role.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.901992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902452 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.902998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.903940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.904948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.905027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80405) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 09:51:50.905183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.905454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.905703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.905949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.906196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.906429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.906679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.906929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.907165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.907396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.907626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.907856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.908106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.908358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.908604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.908925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.909197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.909454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.909692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.909929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.910175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.910413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.910670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.910916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.911154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.911397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.911637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.911883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.912118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.912355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.912589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.912834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.913079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.913332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.913576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.913870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.914139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.connection = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.914385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.914631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.914875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.915117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.915352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.915587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.915822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.916057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.916302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.916537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.916790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.917025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.917294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.917527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.917771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.918001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.918239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.918473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 24 09:51:50.918642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80404) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 24 09:51:50.935370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.935599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.935823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.944542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.944781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.945010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.945105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.955089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.974316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.974538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.974735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:50.974972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.027897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.028046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.079038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.079442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.079558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.079558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.146795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147644 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.147949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.148066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.148277 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.172892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.173019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.173101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.173229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.173354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.173470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.184897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.186266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.197444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.197668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.197887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80405) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198777 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.198907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.199119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.199331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80404) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 24 09:51:51.216833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.217044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 09:51:51.217211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.217402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.217486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 09:51:51.217604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 09:51:51.217776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.217857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.217979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 24 09:51:51.218109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.218243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.218361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.218497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.218612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.218750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.219000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.230830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.231098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.231354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.231625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.232217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.232493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.232760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.233038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.239896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.241225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.243166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.243441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.243715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.243990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.244266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.244520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.244601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.244783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.244865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.245063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.245145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.245368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.245453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.245648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.245730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.245934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.246017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.246275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.246552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.246822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.247092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.247360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.273092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.274709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.275085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.275184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.276833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.277113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.280372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.280606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.280878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.282402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.282678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.282955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.288709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.288984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.291157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.291437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.293456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.293735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.294008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.294394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.294663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.294932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.295197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.295458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.295933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.296329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.296617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.296894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.297166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.297462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.297733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.297999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.314152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.314423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.316845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.317117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.319176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.321815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.322821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.323107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.325498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.325791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.328140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.331440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.332349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.335087 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.335983 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.336275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.336668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.336962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.337237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.337536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.338869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.339162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.339449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.339729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.340022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.340314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.356978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.357251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.357538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.357793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.359862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.360143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.360406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.360666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.365863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.367567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.367835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.368847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.370587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.370869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.370955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.371150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.371425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.371700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.371969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.373935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.374222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.374502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.374788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.375063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.385654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.385926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.386206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.386481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.386757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.387023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.387296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.388813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.389089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.389384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.389673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.389954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.390227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.390507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.406003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.409373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.409570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.412639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.412858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.414474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.415971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.416095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.416371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.416646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.417612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.419218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.419498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.419783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.424000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.426460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.427776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.429403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.429747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.430039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.430316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.430591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.432638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.432925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.433204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.433504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.439642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.439919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.440396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.442729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.443012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.443393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.448587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80405) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.448861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.449133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.449421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.449686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.449948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.450230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.450701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.450978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.451239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.451510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.451766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.452028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.452285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80405) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.452973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80404) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 24 09:51:51.453251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.453549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.453820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.454214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.454707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.455187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.456223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.456703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.457195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.457711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.458097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.458381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.458655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80404) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 24 09:51:51.497073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d42a5a29c0 pid: 80405 (default app) May 24 09:51:51.499474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d42a5a29c0 pid: 80404 (default app) May 24 09:51:51.501558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b043-3549-44ed-8a8c-91a7812e8744 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:51:51.501851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b043-3549-44ed-8a8c-91a7812e8744 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:51:51.502125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221b043-3549-44ed-8a8c-91a7812e8744 None None] PATH_INFO: `/v3` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:51:51.507149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1/1] 10.222.0.35 () {50 vars in 791 bytes} [Tue May 24 09:51:51 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:00.360625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975703da-7238-47f2-8edd-2275089f8d7b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:00.361206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975703da-7238-47f2-8edd-2275089f8d7b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:00.361206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975703da-7238-47f2-8edd-2275089f8d7b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:00.366281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1/2] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:00.372349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:00.372349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:00.372349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:00.440881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: INFO dbcounter [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] Registered counter for database keystone May 24 09:52:00.461865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG oslo_db.sqlalchemy.engines [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80405) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 09:52:00.468074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writer thread running {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 09:52:00.647141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG passlib.handlers.bcrypt [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80405) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 09:52:00.647568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG passlib.handlers.bcrypt [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80405) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 09:52:00.702364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:00.703916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:00.827536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 2/3] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 24 09:52:00.864601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975703da-7238-47f2-8edd-2275089f8d7b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:00.932804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: INFO dbcounter [None req-975703da-7238-47f2-8edd-2275089f8d7b None None] Registered counter for database keystone May 24 09:52:00.952940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG oslo_db.sqlalchemy.engines [None req-975703da-7238-47f2-8edd-2275089f8d7b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80404) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 24 09:52:00.959565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writer thread running {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 24 09:52:01.081762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975703da-7238-47f2-8edd-2275089f8d7b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:01.086086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb7704-10cf-438f-8f32-7a71ffeef513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:01.086756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb7704-10cf-438f-8f32-7a71ffeef513 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:01.087045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb7704-10cf-438f-8f32-7a71ffeef513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:01.087329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb7704-10cf-438f-8f32-7a71ffeef513 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:01.095481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb7704-10cf-438f-8f32-7a71ffeef513 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:01.156822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9afb7704-10cf-438f-8f32-7a71ffeef513 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 09:52:01.157192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb7704-10cf-438f-8f32-7a71ffeef513 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:01.184925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 2/4] 10.222.0.35 () {58 vars in 1324 bytes} [Tue May 24 09:52:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 09:52:01.185080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:01.193206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93563165-9a78-4cef-8235-7bdc7a6f3b82 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:01.197662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1914ee11-ac65-4dfa-b3bc-8ff80223889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:01.198777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ee11-ac65-4dfa-b3bc-8ff80223889c admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:01.199063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ee11-ac65-4dfa-b3bc-8ff80223889c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:01.199345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1914ee11-ac65-4dfa-b3bc-8ff80223889c admin admin] PATH_INFO: `/v3/projects/d39d97f2faca4faba12c96da090b5017` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:01.200871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ee11-ac65-4dfa-b3bc-8ff80223889c admin admin] RBAC: Authorizing `identity:get_project(project_id=d39d97f2faca4faba12c96da090b5017)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:01.363042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1914ee11-ac65-4dfa-b3bc-8ff80223889c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 24 09:52:01.363294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1914ee11-ac65-4dfa-b3bc-8ff80223889c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:01.368068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 3/5] 10.222.0.35 () {56 vars in 1207 bytes} [Tue May 24 09:52:01 2022] GET /identity/v3/projects/d39d97f2faca4faba12c96da090b5017 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:03.627056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b43d-03cc-4465-aa4c-766905297a35 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:03.627607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b43d-03cc-4465-aa4c-766905297a35 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:03.627753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59b43d-03cc-4465-aa4c-766905297a35 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:03.631869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 3/6] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:03.637423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c59b-6568-4de1-bed3-13815aecbf38 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:03.637856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c59b-6568-4de1-bed3-13815aecbf38 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:03.637978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c59b-6568-4de1-bed3-13815aecbf38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:03.656916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8380c59b-6568-4de1-bed3-13815aecbf38 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:03.658444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8380c59b-6568-4de1-bed3-13815aecbf38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:03.666888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 4/7] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 24 09:52:03.689917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b43d-03cc-4465-aa4c-766905297a35 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:03.701883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59b43d-03cc-4465-aa4c-766905297a35 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:03.706194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17422b0a-966e-4542-a1bc-c05dc0ff523b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:03.706826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17422b0a-966e-4542-a1bc-c05dc0ff523b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:03.707113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17422b0a-966e-4542-a1bc-c05dc0ff523b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:03.707395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17422b0a-966e-4542-a1bc-c05dc0ff523b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:03.715047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17422b0a-966e-4542-a1bc-c05dc0ff523b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:03.716830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17422b0a-966e-4542-a1bc-c05dc0ff523b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:03.726173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 4/8] 10.222.0.35 () {58 vars in 1324 bytes} [Tue May 24 09:52:03 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 09:52:03.731380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380c59b-6568-4de1-bed3-13815aecbf38 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:03.740808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380c59b-6568-4de1-bed3-13815aecbf38 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:03.745108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a03bdd-405a-47a2-9be0-1917a5a7392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:03.745990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a03bdd-405a-47a2-9be0-1917a5a7392d admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:03.746273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a03bdd-405a-47a2-9be0-1917a5a7392d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:03.746546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a03bdd-405a-47a2-9be0-1917a5a7392d admin admin] PATH_INFO: `/v3/users/db8edbaf5787451c9cd851df007d0f35` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:03.747913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a03bdd-405a-47a2-9be0-1917a5a7392d admin admin] RBAC: Authorizing `identity:get_user(user_id=db8edbaf5787451c9cd851df007d0f35)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:03.749784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a03bdd-405a-47a2-9be0-1917a5a7392d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:03.751331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 5/9] 10.222.0.35 () {56 vars in 1198 bytes} [Tue May 24 09:52:03 2022] GET /identity/v3/users/db8edbaf5787451c9cd851df007d0f35 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:06.023860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771895-f62b-41d8-9c05-7b983d7bb4b8 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.024403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771895-f62b-41d8-9c05-7b983d7bb4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.024403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1771895-f62b-41d8-9c05-7b983d7bb4b8 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.025775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 5/10] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:06.031009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a979e-6208-4026-8d64-ba89fd08a8ab None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.031407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a979e-6208-4026-8d64-ba89fd08a8ab None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.031567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a979e-6208-4026-8d64-ba89fd08a8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.041395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6689f7-55e3-4f51-9284-0c3df62e7cca None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.041686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6689f7-55e3-4f51-9284-0c3df62e7cca None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.041967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6689f7-55e3-4f51-9284-0c3df62e7cca None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.043301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 6/11] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:06.048279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.048566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.048843 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.050290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-662a979e-6208-4026-8d64-ba89fd08a8ab None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:06.051461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-662a979e-6208-4026-8d64-ba89fd08a8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:06.089097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG passlib.handlers.bcrypt [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80404) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 24 09:52:06.089567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG passlib.handlers.bcrypt [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80404) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 24 09:52:06.131195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 6/12] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:06.136059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:06.137682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:06.153663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a979e-6208-4026-8d64-ba89fd08a8ab None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.178139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:06.239452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 7/13] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:06.257516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a979e-6208-4026-8d64-ba89fd08a8ab None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:06.262149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.293746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:06.294326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.294613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.294894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.362728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa2734-d586-4240-8d73-19e46a57c70e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:06.362934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:06.364372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:06.365167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:06.387365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:06.399085 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:06.400125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.400414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.400693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.412190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:06.423382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:06.424327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:06.426020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 8/14] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:52:06.430896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.461286 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 7/15] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:52:06.466676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.529844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff750a-31be-47f5-98c9-c4af3197662a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:06.562713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:06.563303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.563590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.563683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7ee65-77a3-45ec-8a65-13b6cc21f550 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:06.563870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.564620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:06.564937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:06.567087 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:06.578632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 9/16] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:06.584670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.586129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:06.597784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:06.601541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.601541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.601541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.610150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:06.611617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:06.611732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:52:06.632398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:06.633292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:06.634253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:52:06.634253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:52:06.634253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:52:06.634253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:06.634253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 8/17] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:52:06.639669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.683617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa21a7-1cff-41f9-83fe-f9566bc67231 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:06.717919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:06.718788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.719071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.719341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] PATH_INFO: `/v3/users/db8edbaf5787451c9cd851df007d0f35` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.720898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] RBAC: Authorizing `identity:get_user(user_id=db8edbaf5787451c9cd851df007d0f35)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:06.722982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:06.724593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 10/18] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/users/db8edbaf5787451c9cd851df007d0f35 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:06.730095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.737908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a05f3c-44e3-43b6-b4a7-a009b2157495 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:06.773531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:06.774190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.774472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.774751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.775470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:06.775770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:06.777159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:06.793843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 9/19] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:06.828127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:06.835415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10369358-7125-416d-a7a4-2f7a3851cdfe None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:06.873798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c3de9-90ef-4fcc-82bb-8939549a8608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:06.874620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3de9-90ef-4fcc-82bb-8939549a8608 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:06.874908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3de9-90ef-4fcc-82bb-8939549a8608 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:06.875191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c3de9-90ef-4fcc-82bb-8939549a8608 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:06.876427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c3de9-90ef-4fcc-82bb-8939549a8608 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:06.878762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c3de9-90ef-4fcc-82bb-8939549a8608 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:06.880587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 11/20] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:06.886575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:06.924240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:06.998434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2b006-3a00-43e3-bde8-2d444cd13740 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:07.033547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9f9c7-ee59-480e-bdc0-10aae9c24b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:07.035078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9f9c7-ee59-480e-bdc0-10aae9c24b63 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:07.035473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9f9c7-ee59-480e-bdc0-10aae9c24b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:07.035761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9f9c7-ee59-480e-bdc0-10aae9c24b63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:07.036610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9f9c7-ee59-480e-bdc0-10aae9c24b63 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.domain.id=default, user.id=db8edbaf5787451c9cd851df007d0f35) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:07.036930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9f9c7-ee59-480e-bdc0-10aae9c24b63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:07.038460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9f9c7-ee59-480e-bdc0-10aae9c24b63 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:07.045826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 10/21] 10.222.0.35 () {56 vars in 1322 bytes} [Tue May 24 09:52:06 2022] GET /identity/v3/role_assignments?user.id=db8edbaf5787451c9cd851df007d0f35&scope.domain.id=default&role.id=e1e5746208514068acadbab89907c473 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:09.226858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141687f0-a64b-4cfb-9ffd-73f2f4765833 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:09.227840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141687f0-a64b-4cfb-9ffd-73f2f4765833 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:09.228327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141687f0-a64b-4cfb-9ffd-73f2f4765833 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:09.229990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 12/22] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:09.235589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b2406-4aa1-4142-8db5-e9411bbe0363 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:09.236451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b2406-4aa1-4142-8db5-e9411bbe0363 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:09.237633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b2406-4aa1-4142-8db5-e9411bbe0363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:09.260138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-df5b2406-4aa1-4142-8db5-e9411bbe0363 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:09.261688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-df5b2406-4aa1-4142-8db5-e9411bbe0363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:09.345925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 11/23] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:09.393708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141687f0-a64b-4cfb-9ffd-73f2f4765833 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:09.629795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466720-7159-467f-8d84-4a5563c882c5 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:09.630505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466720-7159-467f-8d84-4a5563c882c5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:09.630998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466720-7159-467f-8d84-4a5563c882c5 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:09.632567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 12/24] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:09.637636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6330fe1-3479-4ee1-b4c8-bfa20f56c8a9 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:09.638162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6330fe1-3479-4ee1-b4c8-bfa20f56c8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:09.638699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6330fe1-3479-4ee1-b4c8-bfa20f56c8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:09.645385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141687f0-a64b-4cfb-9ffd-73f2f4765833 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:09.667741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e6330fe1-3479-4ee1-b4c8-bfa20f56c8a9 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:09.669363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e6330fe1-3479-4ee1-b4c8-bfa20f56c8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:09.717626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:09.729994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:09.730816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:09.731168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:09.731376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:09.763436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:09.765196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:52:09.795281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:09.796167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:09.797434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:52:09.797434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:52:09.797434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:52:09.797434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:09.797434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 13/25] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:09.804073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 13/26] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:09.805176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:09.835999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:09.873401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6330fe1-3479-4ee1-b4c8-bfa20f56c8a9 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:10.027734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aabd61-e327-413c-8078-2c90a738978a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:10.070461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6330fe1-3479-4ee1-b4c8-bfa20f56c8a9 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:10.113442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e834e0b-4da9-4c53-87ae-3090f291a357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:10.114149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e834e0b-4da9-4c53-87ae-3090f291a357 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.114435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e834e0b-4da9-4c53-87ae-3090f291a357 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.114705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e834e0b-4da9-4c53-87ae-3090f291a357 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.115514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e834e0b-4da9-4c53-87ae-3090f291a357 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:10.117012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e834e0b-4da9-4c53-87ae-3090f291a357 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:10.123827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaaa532-2bf4-40d0-be87-f4e9f14313c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:10.124515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaa532-2bf4-40d0-be87-f4e9f14313c3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.124801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaa532-2bf4-40d0-be87-f4e9f14313c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.125080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaaa532-2bf4-40d0-be87-f4e9f14313c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.125865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaaa532-2bf4-40d0-be87-f4e9f14313c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:10.126178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaaa532-2bf4-40d0-be87-f4e9f14313c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:10.127686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaaa532-2bf4-40d0-be87-f4e9f14313c3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:10.147374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 14/27] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:10.148549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064a65-78eb-4c19-ab41-5eec07669b8a None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.149630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064a65-78eb-4c19-ab41-5eec07669b8a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.149630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c064a65-78eb-4c19-ab41-5eec07669b8a None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.153716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 15/28] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:10.154037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064a65-78eb-4c19-ab41-5eec07669b8a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:10.296496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:10.333525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c064a65-78eb-4c19-ab41-5eec07669b8a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:10.372158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43370a-b83f-4dc5-a38c-33c175b546e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:10.373376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43370a-b83f-4dc5-a38c-33c175b546e3 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.373916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43370a-b83f-4dc5-a38c-33c175b546e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.374414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43370a-b83f-4dc5-a38c-33c175b546e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.375757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43370a-b83f-4dc5-a38c-33c175b546e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:10.379029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43370a-b83f-4dc5-a38c-33c175b546e3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:10.380651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 14/29] 10.222.0.35 () {60 vars in 1131 bytes} [Tue May 24 09:52:09 2022] POST /identity/v3/roles => generated 219 bytes in 508 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:52:10.383012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc70a2-dce6-4068-99f6-0f4ac613b269 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.383612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc70a2-dce6-4068-99f6-0f4ac613b269 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.392351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bc70a2-dce6-4068-99f6-0f4ac613b269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.405416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-49bc70a2-dce6-4068-99f6-0f4ac613b269 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:10.407458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-49bc70a2-dce6-4068-99f6-0f4ac613b269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:10.427676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:10.493297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 15/30] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:10.495348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f7e75-2108-4b84-8c83-fd45f9edf96b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.495898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f7e75-2108-4b84-8c83-fd45f9edf96b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.496396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f7e75-2108-4b84-8c83-fd45f9edf96b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.498330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 16/31] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:10.511243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dda45-8167-47cb-a84e-8eeb99107716 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.511896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dda45-8167-47cb-a84e-8eeb99107716 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.512408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72dda45-8167-47cb-a84e-8eeb99107716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.536813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b72dda45-8167-47cb-a84e-8eeb99107716 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:10.539500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b72dda45-8167-47cb-a84e-8eeb99107716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:10.651042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 17/32] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:10.651868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dda45-8167-47cb-a84e-8eeb99107716 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:10.683619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 16/33] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:52:10 2022] POST /identity/v3/projects => generated 301 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:10.715545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2434-5372-47ab-85d3-480d3892b73e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.715891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2434-5372-47ab-85d3-480d3892b73e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.716179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2434-5372-47ab-85d3-480d3892b73e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.717695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 17/34] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:10.718489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f2434-5372-47ab-85d3-480d3892b73e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:10.741138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:10.769572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72dda45-8167-47cb-a84e-8eeb99107716 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:10.834591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f29ac-ec71-4c67-9016-304e001de28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:10.835426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f29ac-ec71-4c67-9016-304e001de28d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.835716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f29ac-ec71-4c67-9016-304e001de28d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.835998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f29ac-ec71-4c67-9016-304e001de28d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.851522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f29ac-ec71-4c67-9016-304e001de28d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9c8f29ac-ec71-4c67-9016-304e001de28d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:10.860802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:10.861943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:10.862907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 18/35] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:10.864049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a12f-aa3b-462e-a127-b5c30dba1369 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.864334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a12f-aa3b-462e-a127-b5c30dba1369 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.864603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a12f-aa3b-462e-a127-b5c30dba1369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.878811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394f2434-5372-47ab-85d3-480d3892b73e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:10.886404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e000a12f-aa3b-462e-a127-b5c30dba1369 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:10.887742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e000a12f-aa3b-462e-a127-b5c30dba1369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:10.911472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:10.920850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:10.921616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:10.921904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:10.922180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:10.922934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:10.925035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:10.973709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 19/36] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:10.974916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000a12f-aa3b-462e-a127-b5c30dba1369 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:11.002630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 18/37] 10.222.0.35 () {60 vars in 1131 bytes} [Tue May 24 09:52:10 2022] POST /identity/v3/roles => generated 225 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:11.003690 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:11.073593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:11.123440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000a12f-aa3b-462e-a127-b5c30dba1369 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:11.129618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37364-c838-4f5a-928e-a3c0bf1e387d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:11.178907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:11.179629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.179918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.180203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.180992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:11.181681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:11.181800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:11.182481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.182764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.183044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.183817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:11.184467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:11.185366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:11.200190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 20/38] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:11.206816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:11.262843 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 19/39] 10.222.0.35 () {60 vars in 1131 bytes} [Tue May 24 09:52:11 2022] POST /identity/v3/roles => generated 223 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:11.331265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7a829-0ec1-4b6e-a398-80db3961b851 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:11.365460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:11.372465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a538f0-5398-4fac-bf2f-e6af79d7d1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:11.373471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a538f0-5398-4fac-bf2f-e6af79d7d1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.373730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a538f0-5398-4fac-bf2f-e6af79d7d1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.374010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a538f0-5398-4fac-bf2f-e6af79d7d1b3 None admin] PATH_INFO: `/v3/users/db8edbaf5787451c9cd851df007d0f35` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.376680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a538f0-5398-4fac-bf2f-e6af79d7d1b3 None admin] RBAC: Authorizing `identity:get_user(user_id=db8edbaf5787451c9cd851df007d0f35)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:11.376840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a538f0-5398-4fac-bf2f-e6af79d7d1b3 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:11.378540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 21/40] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:11 2022] GET /identity/v3/users/db8edbaf5787451c9cd851df007d0f35 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:11.384044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:11.413122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a16ed-059d-4a99-b1aa-a37f131f6af6 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.413727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a16ed-059d-4a99-b1aa-a37f131f6af6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.413727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a16ed-059d-4a99-b1aa-a37f131f6af6 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.415124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 22/41] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:11.422487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707fc15-f805-460a-979a-4cc8ea0c1b3e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.422754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707fc15-f805-460a-979a-4cc8ea0c1b3e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.422978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3707fc15-f805-460a-979a-4cc8ea0c1b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.440763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:11.441775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-3707fc15-f805-460a-979a-4cc8ea0c1b3e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:11.443035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3707fc15-f805-460a-979a-4cc8ea0c1b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:11.507324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467623a-cf8c-4b4e-85c0-b5d75afabd7b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:11.521182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 23/42] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:11.543082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:11.543889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.544255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.544547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.545205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707fc15-f805-460a-979a-4cc8ea0c1b3e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:11.545847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:11.547312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:11.549055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 20/43] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:11.554515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:11.647252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3707fc15-f805-460a-979a-4cc8ea0c1b3e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:11.652068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a438bb4-e2c5-4fdb-9156-162e8ed60927 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:11.665719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:11.677103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:11.684922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:11.685078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb30bf0-6b01-4a10-97ba-1d462b7158c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:11.685789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.685950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb30bf0-6b01-4a10-97ba-1d462b7158c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.686078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.686238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb30bf0-6b01-4a10-97ba-1d462b7158c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.686340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.686520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb30bf0-6b01-4a10-97ba-1d462b7158c6 None admin] PATH_INFO: `/v3/domains/default/users/db8edbaf5787451c9cd851df007d0f35/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.687550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:11.688759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb30bf0-6b01-4a10-97ba-1d462b7158c6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=db8edbaf5787451c9cd851df007d0f35, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:11.689026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:11.690241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb30bf0-6b01-4a10-97ba-1d462b7158c6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:11.691184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 24/44] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:11.696722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:11.762947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 21/45] 10.222.0.35 () {58 vars in 1361 bytes} [Tue May 24 09:52:11 2022] PUT /identity/v3/domains/default/users/db8edbaf5787451c9cd851df007d0f35/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:52:11.797186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fddb72-e317-4aa4-85f1-c04e01d0a690 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:11.834346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db9d6b-802f-4d5d-a11c-c890e0f88a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:11.835026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db9d6b-802f-4d5d-a11c-c890e0f88a31 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:11.835312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db9d6b-802f-4d5d-a11c-c890e0f88a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:11.835591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db9d6b-802f-4d5d-a11c-c890e0f88a31 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:11.836321 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db9d6b-802f-4d5d-a11c-c890e0f88a31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:11.837829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db9d6b-802f-4d5d-a11c-c890e0f88a31 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:11.925928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 25/46] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:52:11 2022] POST /identity/v3/projects => generated 312 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:52:13.948307 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e06462-0a62-4b31-9126-ef4b41e0aa2e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:13.948835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e06462-0a62-4b31-9126-ef4b41e0aa2e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:13.948943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e06462-0a62-4b31-9126-ef4b41e0aa2e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:13.950426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 22/47] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:13.956242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681ecf0-83c3-453f-9bad-82adf248eefb None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:13.956686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681ecf0-83c3-453f-9bad-82adf248eefb None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:13.956776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681ecf0-83c3-453f-9bad-82adf248eefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:13.974377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f681ecf0-83c3-453f-9bad-82adf248eefb None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:13.975625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f681ecf0-83c3-453f-9bad-82adf248eefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:14.023799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:14.046012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 26/48] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:14.069422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e06462-0a62-4b31-9126-ef4b41e0aa2e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:14.172662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e06462-0a62-4b31-9126-ef4b41e0aa2e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:14.179388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb24a15-d344-438b-8008-f6550fdc55fe None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.179746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb24a15-d344-438b-8008-f6550fdc55fe None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.180075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb24a15-d344-438b-8008-f6550fdc55fe None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.181573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 27/49] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:14.186573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907d133-a303-4e1e-9149-67ee5acbcd60 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.186943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907d133-a303-4e1e-9149-67ee5acbcd60 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.187301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907d133-a303-4e1e-9149-67ee5acbcd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.204090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c907d133-a303-4e1e-9149-67ee5acbcd60 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:14.205222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c907d133-a303-4e1e-9149-67ee5acbcd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:14.206288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:14.207095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.207380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.207829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.218909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:14.227109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:14.227996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:14.228884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 23/50] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:14.233880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:14.267807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:14.275605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 28/51] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:14.300396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907d133-a303-4e1e-9149-67ee5acbcd60 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:14.339374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43759a2-8d4d-4166-9f21-1fd73bfafe28 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:14.374691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:14.375367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.375655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.375929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.376681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:14.376996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:14.378997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:14.387099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 24/52] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:14.392803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:14.400543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907d133-a303-4e1e-9149-67ee5acbcd60 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:14.410366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:14.434683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:14.435487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.435779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.436061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.437302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:14.438775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:14.440909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 29/53] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:14.446167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:14.487224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:14.493202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f63fc-b07b-4483-ba27-d14dcaed7964 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:14.527599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:14.528478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.528763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.529042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] PATH_INFO: `/v3/users/db8edbaf5787451c9cd851df007d0f35` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.530273 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] RBAC: Authorizing `identity:get_user(user_id=db8edbaf5787451c9cd851df007d0f35)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:14.531734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:14.533529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 25/54] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:14 2022] GET /identity/v3/users/db8edbaf5787451c9cd851df007d0f35 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:14.539233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:14.546140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca5793-0c60-434f-91dd-46706f61edf8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:14.580507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:14.581726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.582012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.582292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.583009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:14.584473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:14.637762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb876e8-ab9a-4c18-96b1-e0f91482986f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:14.667465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 30/55] 10.222.0.35 () {60 vars in 1140 bytes} [Tue May 24 09:52:14 2022] POST /identity/v3/projects => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:52:14.673726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7ec9f-62f4-4ddf-bb13-b44d95aaef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:14.674702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7ec9f-62f4-4ddf-bb13-b44d95aaef8e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.675185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7ec9f-62f4-4ddf-bb13-b44d95aaef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.675643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7ec9f-62f4-4ddf-bb13-b44d95aaef8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.676994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7ec9f-62f4-4ddf-bb13-b44d95aaef8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:14.678538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7ec9f-62f4-4ddf-bb13-b44d95aaef8e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:14.680341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 26/56] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:14.686445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:14.717916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:14.797177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b249b-1925-479d-91ee-265e42621de9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:14.834956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761e1d6-7d72-49b8-bc42-ef7f151b1123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:14.835643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761e1d6-7d72-49b8-bc42-ef7f151b1123 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:14.835933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761e1d6-7d72-49b8-bc42-ef7f151b1123 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:14.836219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761e1d6-7d72-49b8-bc42-ef7f151b1123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:14.837077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761e1d6-7d72-49b8-bc42-ef7f151b1123 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.domain.id=default, user.id=db8edbaf5787451c9cd851df007d0f35) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:14.837431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761e1d6-7d72-49b8-bc42-ef7f151b1123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:14.838912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761e1d6-7d72-49b8-bc42-ef7f151b1123 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:14.845239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 31/57] 10.222.0.35 () {56 vars in 1322 bytes} [Tue May 24 09:52:14 2022] GET /identity/v3/role_assignments?user.id=db8edbaf5787451c9cd851df007d0f35&scope.domain.id=default&role.id=e1e5746208514068acadbab89907c473 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:16.859744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a6aa-b31e-4f93-b85b-817d7aafd4b5 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:16.860297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a6aa-b31e-4f93-b85b-817d7aafd4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:16.860297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15a6aa-b31e-4f93-b85b-817d7aafd4b5 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:16.861697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 27/58] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:16.867463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e71ff2-d6bf-4c2e-9c3b-15eb9e2f9444 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:16.867890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e71ff2-d6bf-4c2e-9c3b-15eb9e2f9444 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:16.868020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e71ff2-d6bf-4c2e-9c3b-15eb9e2f9444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:16.885016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-97e71ff2-d6bf-4c2e-9c3b-15eb9e2f9444 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:16.886245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-97e71ff2-d6bf-4c2e-9c3b-15eb9e2f9444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:16.950349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 32/59] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:16.975229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15a6aa-b31e-4f93-b85b-817d7aafd4b5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:17.006498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:17.076227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15a6aa-b31e-4f93-b85b-817d7aafd4b5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:17.109655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690c7f0-f53c-4576-a9ab-a9e9a7c2f2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:17.110487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690c7f0-f53c-4576-a9ab-a9e9a7c2f2ba None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:17.110724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690c7f0-f53c-4576-a9ab-a9e9a7c2f2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:17.111003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690c7f0-f53c-4576-a9ab-a9e9a7c2f2ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:17.112962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690c7f0-f53c-4576-a9ab-a9e9a7c2f2ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:17.114557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690c7f0-f53c-4576-a9ab-a9e9a7c2f2ba None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:17.116160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 28/60] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:17.121680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e71ff2-d6bf-4c2e-9c3b-15eb9e2f9444 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:17.145008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:17.218938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e71ff2-d6bf-4c2e-9c3b-15eb9e2f9444 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:17.252277 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e15d6d-0cc6-48ed-abed-bbe2e1d0c41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:17.253012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e15d6d-0cc6-48ed-abed-bbe2e1d0c41e None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:17.253312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e15d6d-0cc6-48ed-abed-bbe2e1d0c41e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:17.253590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e15d6d-0cc6-48ed-abed-bbe2e1d0c41e None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:17.254330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e15d6d-0cc6-48ed-abed-bbe2e1d0c41e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:17.255785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e15d6d-0cc6-48ed-abed-bbe2e1d0c41e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:17.374126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 33/61] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:52:17 2022] POST /identity/v3/users => generated 275 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:20.799243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e4836-11aa-4e3a-bdfc-62bdbb0665de None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:20.800758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e4836-11aa-4e3a-bdfc-62bdbb0665de None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:20.801256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e4836-11aa-4e3a-bdfc-62bdbb0665de None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:20.802895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 29/62] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:20.808726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c1f7b-2a0a-42a6-b27b-a23c502baf67 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:20.809623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c1f7b-2a0a-42a6-b27b-a23c502baf67 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:20.810107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c1f7b-2a0a-42a6-b27b-a23c502baf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:20.832941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bf32-526a-4dd9-9950-dbc53bac8391 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:20.833593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bf32-526a-4dd9-9950-dbc53bac8391 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:20.834101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7bf32-526a-4dd9-9950-dbc53bac8391 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:20.835763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 30/63] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:20.840838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877753b0-1b73-41de-9a70-e788ef11b9d6 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:20.840956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b02c1f7b-2a0a-42a6-b27b-a23c502baf67 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:20.841486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877753b0-1b73-41de-9a70-e788ef11b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:20.842011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877753b0-1b73-41de-9a70-e788ef11b9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:20.842145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b02c1f7b-2a0a-42a6-b27b-a23c502baf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:20.873252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-877753b0-1b73-41de-9a70-e788ef11b9d6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:20.876281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-877753b0-1b73-41de-9a70-e788ef11b9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:20.965361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 34/64] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:20.967121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7208f2-f690-48dc-8524-26d9a692fce5 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:20.968595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7208f2-f690-48dc-8524-26d9a692fce5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:20.969245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7208f2-f690-48dc-8524-26d9a692fce5 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:20.972216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 35/65] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:20.980852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbde53b-381b-4a60-8f56-218d9e38fcfc None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:20.981468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbde53b-381b-4a60-8f56-218d9e38fcfc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:20.981949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbde53b-381b-4a60-8f56-218d9e38fcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:20.988290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 31/66] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:20.994258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877753b0-1b73-41de-9a70-e788ef11b9d6 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.009894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-abbde53b-381b-4a60-8f56-218d9e38fcfc None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:21.011414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-abbde53b-381b-4a60-8f56-218d9e38fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:21.013724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.045198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.101487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 36/67] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:21.102711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbde53b-381b-4a60-8f56-218d9e38fcfc None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.161718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877753b0-1b73-41de-9a70-e788ef11b9d6 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.207923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.208742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.209036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.209606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.224473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:21.234956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:21.235825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:21.236592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 32/68] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:21.237788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.251911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbde53b-381b-4a60-8f56-218d9e38fcfc None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.304508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.305731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.306104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.306464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.316690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.321032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:21.329211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:21.330235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:21.331644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 37/69] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:21.332277 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.342172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.355833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bac69-0c6b-493c-a75c-ca216464f760 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.385788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2962ea-58dc-4c2a-b8d6-d6f94a7e882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.386581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2962ea-58dc-4c2a-b8d6-d6f94a7e882f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.386871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2962ea-58dc-4c2a-b8d6-d6f94a7e882f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.387153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2962ea-58dc-4c2a-b8d6-d6f94a7e882f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.398187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2962ea-58dc-4c2a-b8d6-d6f94a7e882f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-7e2962ea-58dc-4c2a-b8d6-d6f94a7e882f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:21.405848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:21.406702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:21.407479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 33/70] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:21.408251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816489-718b-4d3d-83b8-6ed4e73a8bf1 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.408538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816489-718b-4d3d-83b8-6ed4e73a8bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.408931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f816489-718b-4d3d-83b8-6ed4e73a8bf1 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.411008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 34/71] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:21.411969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeff663-a2b6-4f66-98f8-7ce21d8461d4 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.412261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeff663-a2b6-4f66-98f8-7ce21d8461d4 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.412545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeff663-a2b6-4f66-98f8-7ce21d8461d4 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.414035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 35/72] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:21.414760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeff663-a2b6-4f66-98f8-7ce21d8461d4 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.435485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f1b10-f38b-4b15-a3a2-964e52430fa2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.468645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.469350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.469636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.469916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.470705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:21.471012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:21.472582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:21.480607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 38/73] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:21.480607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.519043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeff663-a2b6-4f66-98f8-7ce21d8461d4 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.541153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.552621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cb503-b288-4c0f-a7a2-572901caa9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.553323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb503-b288-4c0f-a7a2-572901caa9bd None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.553611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb503-b288-4c0f-a7a2-572901caa9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.553897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cb503-b288-4c0f-a7a2-572901caa9bd None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.554657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb503-b288-4c0f-a7a2-572901caa9bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:21.554962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb503-b288-4c0f-a7a2-572901caa9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:21.556365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cb503-b288-4c0f-a7a2-572901caa9bd None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:21.564274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 36/74] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:21.569103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f696d75-09dc-493c-8de9-646e26198ea2 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.569103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f696d75-09dc-493c-8de9-646e26198ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.569103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f696d75-09dc-493c-8de9-646e26198ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.581629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4c707-6c1a-4553-841c-5e75c02ea672 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.582895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-3f696d75-09dc-493c-8de9-646e26198ea2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:21.584111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3f696d75-09dc-493c-8de9-646e26198ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:21.591330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.615805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49d13f-a17f-4c0a-bf19-0793ff9d0485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.616404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49d13f-a17f-4c0a-bf19-0793ff9d0485 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.616687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49d13f-a17f-4c0a-bf19-0793ff9d0485 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.616965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49d13f-a17f-4c0a-bf19-0793ff9d0485 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.617762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49d13f-a17f-4c0a-bf19-0793ff9d0485 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:21.618071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49d13f-a17f-4c0a-bf19-0793ff9d0485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:21.619464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49d13f-a17f-4c0a-bf19-0793ff9d0485 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:21.627329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 39/75] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:21.629883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f9c8-d10d-4e7c-93ef-e7dde48bfec2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.630180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f9c8-d10d-4e7c-93ef-e7dde48bfec2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.630469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05f9c8-d10d-4e7c-93ef-e7dde48bfec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.646493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-cc05f9c8-d10d-4e7c-93ef-e7dde48bfec2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:21.647684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cc05f9c8-d10d-4e7c-93ef-e7dde48bfec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:21.651308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 37/76] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:21.652145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f696d75-09dc-493c-8de9-646e26198ea2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.727289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 40/77] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:21.728140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05f9c8-d10d-4e7c-93ef-e7dde48bfec2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.763658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f696d75-09dc-493c-8de9-646e26198ea2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.781771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.798871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.800469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.801056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.801056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] PATH_INFO: `/v3/users/db8edbaf5787451c9cd851df007d0f35` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.802276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] RBAC: Authorizing `identity:get_user(user_id=db8edbaf5787451c9cd851df007d0f35)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:21.803712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:21.805339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 38/78] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/users/db8edbaf5787451c9cd851df007d0f35 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:21.806226 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.818072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:21.830311 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05f9c8-d10d-4e7c-93ef-e7dde48bfec2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.865624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.866562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.866787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.867071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.881030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:21.882658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:21.884344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 41/79] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:21.885112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.907805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a01d499-069e-44e3-b98b-7acf94f30069 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.941469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:21.942339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:21.942624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:21.942899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:21.944100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:21.945589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:21.947210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 39/80] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:21.947967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:21.984309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a9258-5bdb-4ab6-85a7-5dc058dc7714 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:21.994705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.017227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.018086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.018954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.019477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.028640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.029659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:22.037730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:22.038618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:22.039386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 42/81] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:22.040056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.052709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d1b76-bcea-40ad-8ae0-2b3b3842bbe7 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.087136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.087933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.088220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.088502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.089710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.091284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.092894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 40/82] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:22.093677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.139309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e07e3-bb01-4564-ad0d-257e1578e4fc None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.176346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.177046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.177358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.177641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.199835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed2015-6944-44b5-bbf2-58d3484d0ace None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.217446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.233219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.233902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.234189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.234467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.245058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.246563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.249826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.303590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.305076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.343071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 43/83] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:22.343718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.406525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 41/84] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:22.410505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.419960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.449734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d731317-d7ed-4318-9eb4-e8c8e1511d21 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.457068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.484350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.485006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.485324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.485609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.511722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac040b06-b09e-4a81-a1a2-aa5270529304 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.545748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.546408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.546692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.546968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.547713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] RBAC: Adding query filter params (name=member) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:22.548015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.549542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.553866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.555342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.557407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 42/85] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:22.558174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.625452 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.655809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c25745-c4d7-457d-a618-30253d85484c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.658046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 44/86] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:22.659429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.669908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.694719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.695394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.695680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.695964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.696702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.698622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.758434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352b8d8-926c-446b-925e-0885f9ef929d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.763619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 43/87] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:52:22 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:22.764799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.812741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.813650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.814275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.814579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.816227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.817773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.819568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 45/88] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:22.822539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.872498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.879994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67360296-6150-4998-ae1c-cc81936200d1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.920948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.921895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.922186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.922559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.923828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.925407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.927498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 44/89] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:22.928203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:22.948911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701445d-3e31-4dda-8dfc-e4a71a1bf9f4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:22.962376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:22.992003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:22.992899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:22.993185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:22.993499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:22.994724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:22.996248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:22.998134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 46/90] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:23.002184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:23.052649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321867d-5cc6-4ab0-9049-40a5a85fec2a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:23.094647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:23.095591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:23.095882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:23.096163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:23.098256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:23.099734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:23.104237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 45/91] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:23.104237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:23.137132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea58ef-ea8f-4dd7-8108-37b9980caaa4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:23.148458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:23.184504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:23.185733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:23.186161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:23.186603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:23.187591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=db8edbaf5787451c9cd851df007d0f35) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:23.188046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:23.191038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:23.203075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 47/92] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:22 2022] GET /identity/v3/role_assignments?user.id=db8edbaf5787451c9cd851df007d0f35&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:23.204273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:23.235450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a095d-838d-48e1-afd9-19d9a6af2052 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:23.261699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:23.281429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:23.282320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:23.282726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:23.283112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:23.285213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] RBAC: Adding query filter params (role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=90d9e3351e21469a95b1a88347c81225) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:23.285740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:23.287902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:23.297581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 46/93] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:23 2022] GET /identity/v3/role_assignments?user.id=90d9e3351e21469a95b1a88347c81225&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:23.300113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:23.463254 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b48a6f-9ed0-4164-a656-f388c8768c61 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:23.495612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e0f257-359f-429c-aca5-05409ea688bb None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:23.500687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:23.501382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:23.501678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:23.501962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:23.503171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=90d9e3351e21469a95b1a88347c81225) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:23.503553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:23.505163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:23.512079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 47/94] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:23 2022] GET /identity/v3/role_assignments?user.id=90d9e3351e21469a95b1a88347c81225&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:23.554204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:23.554893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:23.555182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:23.555465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:23.620206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:23.669073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:23.670815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:23.809433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 48/95] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:23.815038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:23.856859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:23.939290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482afdcf-f498-4141-80b8-87fa380bd27d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:23.975891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b8acc-7ee9-4df1-bca1-151f7f68ee6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:23.976765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b8acc-7ee9-4df1-bca1-151f7f68ee6f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:23.977055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b8acc-7ee9-4df1-bca1-151f7f68ee6f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:23.977358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b8acc-7ee9-4df1-bca1-151f7f68ee6f None admin] PATH_INFO: `/v3/projects/d4686cf3d9244e29ba9af47e9112864c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:23.978560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b8acc-7ee9-4df1-bca1-151f7f68ee6f None admin] RBAC: Authorizing `identity:get_project(project_id=d4686cf3d9244e29ba9af47e9112864c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:23.980039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b8acc-7ee9-4df1-bca1-151f7f68ee6f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:23.982150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 48/96] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:23 2022] GET /identity/v3/projects/d4686cf3d9244e29ba9af47e9112864c => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:23.988117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:24.109592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:24.157486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8565f4-b301-4181-9662-21286dfb482a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:24.210812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4c386-6de4-467b-a110-1029323dbea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:24.211593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c386-6de4-467b-a110-1029323dbea6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:24.211883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c386-6de4-467b-a110-1029323dbea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:24.212159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4c386-6de4-467b-a110-1029323dbea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:24.213002 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4c386-6de4-467b-a110-1029323dbea6 None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=d4686cf3d9244e29ba9af47e9112864c, user.id=90d9e3351e21469a95b1a88347c81225) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:24.213334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4c386-6de4-467b-a110-1029323dbea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:24.214806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c4c386-6de4-467b-a110-1029323dbea6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:24.225602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 49/97] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:23 2022] GET /identity/v3/role_assignments?user.id=90d9e3351e21469a95b1a88347c81225&scope.project.id=d4686cf3d9244e29ba9af47e9112864c&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:26.024433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6d938-406a-4f6d-97fd-a52a93999694 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.025072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6d938-406a-4f6d-97fd-a52a93999694 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.025072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6d938-406a-4f6d-97fd-a52a93999694 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.026453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 49/98] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:26.032531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be496a4-9420-4d48-9f62-18c92df6dba6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.033557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be496a4-9420-4d48-9f62-18c92df6dba6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.034609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be496a4-9420-4d48-9f62-18c92df6dba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.062513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0be496a4-9420-4d48-9f62-18c92df6dba6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:26.064079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0be496a4-9420-4d48-9f62-18c92df6dba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:26.155482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 50/99] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:26.189736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6d938-406a-4f6d-97fd-a52a93999694 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:26.297676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee9a5-cf6f-4867-a9a3-3c2866249571 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.298295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee9a5-cf6f-4867-a9a3-3c2866249571 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.298780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dee9a5-cf6f-4867-a9a3-3c2866249571 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.300835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 51/100] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:26.306301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636a89-2684-4ba3-a9aa-0ebf68cd60f2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.306902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636a89-2684-4ba3-a9aa-0ebf68cd60f2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.307384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09636a89-2684-4ba3-a9aa-0ebf68cd60f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.320241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:26.330815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6d938-406a-4f6d-97fd-a52a93999694 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:26.346344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-09636a89-2684-4ba3-a9aa-0ebf68cd60f2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:26.347560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-09636a89-2684-4ba3-a9aa-0ebf68cd60f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:26.380052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9c365-a231-4468-9148-78e56989555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:26.380865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9c365-a231-4468-9148-78e56989555a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.381153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9c365-a231-4468-9148-78e56989555a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.381516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9c365-a231-4468-9148-78e56989555a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.400113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9c365-a231-4468-9148-78e56989555a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-faa9c365-a231-4468-9148-78e56989555a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:26.415490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:26.416428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:26.417747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 50/101] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:26.422504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9c365-a231-4468-9148-78e56989555a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:26.485671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:26.504427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 52/102] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:26.526272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65ce07-300e-4992-b075-a9283ed84f6b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.526783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65ce07-300e-4992-b075-a9283ed84f6b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.527230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c65ce07-300e-4992-b075-a9283ed84f6b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.528768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 53/103] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:26.533998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64655c-878d-431b-a5e8-6088b066f626 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.534482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64655c-878d-431b-a5e8-6088b066f626 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.534916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de64655c-878d-431b-a5e8-6088b066f626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.571443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-de64655c-878d-431b-a5e8-6088b066f626 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:26.572560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-de64655c-878d-431b-a5e8-6088b066f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:26.589303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9c365-a231-4468-9148-78e56989555a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:26.639498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:26.640430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.640716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.640996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.641799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:26.642113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:26.644080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:26.652801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 51/104] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:26.653855 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:26.673207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 54/105] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:26.674218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:26.676857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3ed2-042d-4c98-b53e-b200892183f3 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.677377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3ed2-042d-4c98-b53e-b200892183f3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.677947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3ed2-042d-4c98-b53e-b200892183f3 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.679435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 55/106] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:26.680599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c3ed2-042d-4c98-b53e-b200892183f3 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:26.763012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee18f5-24a6-4e8a-9ba1-4a6d2c31cd54 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:26.800746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca1326-41ef-49be-9283-db0db6f72a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:26.801591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca1326-41ef-49be-9283-db0db6f72a9b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.801880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca1326-41ef-49be-9283-db0db6f72a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.802156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca1326-41ef-49be-9283-db0db6f72a9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.811483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c3ed2-042d-4c98-b53e-b200892183f3 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:26.812822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca1326-41ef-49be-9283-db0db6f72a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-98ca1326-41ef-49be-9283-db0db6f72a9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:26.821662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:26.822591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:26.823353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:26.825730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 52/107] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:52:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:26.827122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc11ba-c3c0-4804-881b-d876ca8c9087 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.827599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc11ba-c3c0-4804-881b-d876ca8c9087 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.828061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc11ba-c3c0-4804-881b-d876ca8c9087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.844047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:26.845499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-aefc11ba-c3c0-4804-881b-d876ca8c9087 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:26.846925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-aefc11ba-c3c0-4804-881b-d876ca8c9087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:26.857226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:26.858673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.859112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.859397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] PATH_INFO: `/v3/users/fef81a30ac58471b96a6d34f76d4e10d` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.876241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] RBAC: Authorizing `identity:get_user(user_id=fef81a30ac58471b96a6d34f76d4e10d)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:26.877823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:26.879695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 56/108] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:26 2022] GET /identity/v3/users/fef81a30ac58471b96a6d34f76d4e10d => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:26.880612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:26.920355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 53/109] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:26.922092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97812f-9527-409e-9d42-6de0e4ecf188 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:26.922389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97812f-9527-409e-9d42-6de0e4ecf188 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:26.922668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97812f-9527-409e-9d42-6de0e4ecf188 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:26.923913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 54/110] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:26.925134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97812f-9527-409e-9d42-6de0e4ecf188 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:26.944883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.004307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9ffb1-466c-4dfd-928b-e26abd89fd7a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.043442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.044397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.044988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.045432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.048148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97812f-9527-409e-9d42-6de0e4ecf188 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.058066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:27.067227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:27.068105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:27.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 57/111] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:27.070551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.085086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dde49-f417-4805-bb89-70d420aaf412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.086002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dde49-f417-4805-bb89-70d420aaf412 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.086453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dde49-f417-4805-bb89-70d420aaf412 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.086915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dde49-f417-4805-bb89-70d420aaf412 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.087676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dde49-f417-4805-bb89-70d420aaf412 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:27.088152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dde49-f417-4805-bb89-70d420aaf412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.089578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dde49-f417-4805-bb89-70d420aaf412 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.098443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 55/112] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:52:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:27.099945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccda4e-87c4-44cd-9f7b-8f8f589f9ff3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.100241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccda4e-87c4-44cd-9f7b-8f8f589f9ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.100526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccda4e-87c4-44cd-9f7b-8f8f589f9ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.118072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-7dccda4e-87c4-44cd-9f7b-8f8f589f9ff3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:27.119401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7dccda4e-87c4-44cd-9f7b-8f8f589f9ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:27.120533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.185356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8966dc-15a6-4ecb-a1f1-8b22bddb6478 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.191818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.196693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 56/113] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:27.210286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccda4e-87c4-44cd-9f7b-8f8f589f9ff3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.221898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.222796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.223237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.223661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.297984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.300300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.322360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccda4e-87c4-44cd-9f7b-8f8f589f9ff3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.343022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.361707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.363474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.363758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.364045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.365238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.366724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.368351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 57/114] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:27.369303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.416977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 58/115] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:27.416977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.442808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.480299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce8acbe-d9b3-478d-ad83-ef19ad90178e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.519624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.520313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.520604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.520886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.521693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:27.522005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.523425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.529136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51518ae-8762-43dd-82ea-f0c166dbd371 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.531924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 58/116] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:27.531924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.570220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.571238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.571531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.571811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.573028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.574522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.581626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 59/117] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:27.581626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.595130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.637258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0a065-0421-4b8b-9db9-bfb603b958cf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.667054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.673669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd969135-88de-472c-9812-de0a694f538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.675661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd969135-88de-472c-9812-de0a694f538c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.675947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd969135-88de-472c-9812-de0a694f538c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.676504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd969135-88de-472c-9812-de0a694f538c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.686797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd969135-88de-472c-9812-de0a694f538c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:27.687088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d41cb7-012a-41bc-8f81-379fc9b50369 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-dd969135-88de-472c-9812-de0a694f538c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:27.695670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:27.696524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:27.697347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 59/118] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:27.702108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd969135-88de-472c-9812-de0a694f538c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.720667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.721872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.722181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.722540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.723317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.724793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.794262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 60/119] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:52:27 2022] POST /identity/v3/projects => generated 302 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:27.795098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.813795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd969135-88de-472c-9812-de0a694f538c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.856351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.857162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.857479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.857765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.859075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.860666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.862840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 60/120] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:27.863462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.866363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.914608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02871b1a-1cb5-4bb6-be25-0cc6ddf0267c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:27.934318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:27.951801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8110d780-46ba-4c62-a3ee-450d823f0a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:27.952714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d780-46ba-4c62-a3ee-450d823f0a3f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.953007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d780-46ba-4c62-a3ee-450d823f0a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.953306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8110d780-46ba-4c62-a3ee-450d823f0a3f None admin] PATH_INFO: `/v3/users/db8edbaf5787451c9cd851df007d0f35` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.954566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110d780-46ba-4c62-a3ee-450d823f0a3f None admin] RBAC: Authorizing `identity:get_user(user_id=db8edbaf5787451c9cd851df007d0f35)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:27.956787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8110d780-46ba-4c62-a3ee-450d823f0a3f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:27.958558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 61/121] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/users/db8edbaf5787451c9cd851df007d0f35 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:27.959842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c413e-cf38-45f0-9b60-6403c1374dcf None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:27.960160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c413e-cf38-45f0-9b60-6403c1374dcf None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:27.960446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c413e-cf38-45f0-9b60-6403c1374dcf None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:27.961951 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 62/122] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:27.962565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c413e-cf38-45f0-9b60-6403c1374dcf None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:27.988792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8950272a-f60a-4a0c-879d-6bd5a7cbe714 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.029794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.030701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.030988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.031264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.032462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:28.034063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:28.036031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 61/123] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:28.036648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.085155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c413e-cf38-45f0-9b60-6403c1374dcf None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.113501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:28.118814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9eb068-59f1-469e-bdec-41982715c5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.119492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9eb068-59f1-469e-bdec-41982715c5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.119780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9eb068-59f1-469e-bdec-41982715c5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.120096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9eb068-59f1-469e-bdec-41982715c5c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.120854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9eb068-59f1-469e-bdec-41982715c5c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:28.121161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9eb068-59f1-469e-bdec-41982715c5c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:28.122593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9eb068-59f1-469e-bdec-41982715c5c7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:28.132033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 63/124] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:28.133390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bfbed-99e9-4cd6-ba3f-3d0cb5113e27 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.133681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bfbed-99e9-4cd6-ba3f-3d0cb5113e27 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.134002 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bfbed-99e9-4cd6-ba3f-3d0cb5113e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.154502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-959bfbed-99e9-4cd6-ba3f-3d0cb5113e27 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:28.155775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-959bfbed-99e9-4cd6-ba3f-3d0cb5113e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:28.163578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1008eae7-fcf4-4e19-b1b4-2ea9367e9a7f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.194478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:28.203309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.204059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.204351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.204633 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.206154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] RBAC: Adding query filter params (role.id=1268b0be014043318348a2854238939a, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=fef81a30ac58471b96a6d34f76d4e10d) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:28.206479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:28.208073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:28.225239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 62/125] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:28 2022] GET /identity/v3/role_assignments?user.id=fef81a30ac58471b96a6d34f76d4e10d&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=1268b0be014043318348a2854238939a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:28.225239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.253131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 64/126] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:28.254971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bfbed-99e9-4cd6-ba3f-3d0cb5113e27 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.371047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e653a8-b930-4ad1-b69e-12a568f8788e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.385301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bfbed-99e9-4cd6-ba3f-3d0cb5113e27 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.418541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:28.426882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.430607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.430607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.430607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.431667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:28.433636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:28.435358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.436200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 63/127] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:28 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:28.436303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.441658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.441658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/90d9e3351e21469a95b1a88347c81225/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.441658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] RBAC: Authorizing `identity:create_grant(project_id=638fa7b1962d4ab7bf021ee0225dd847, user_id=90d9e3351e21469a95b1a88347c81225, role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:28.441658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:28.442984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.505305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:28.525961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 65/128] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:28 2022] PUT /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/90d9e3351e21469a95b1a88347c81225/roles/60347b7e855240eb9c04a1cb202a3500 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:28.526819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.554340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674c57e5-3e92-457c-b390-70c0a8535d21 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.595353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.596929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.597492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.597984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.599761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:28.601539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:28.604380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 64/129] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:28 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:28.605868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.724054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d8060-b140-447b-920c-8c8816a8b694 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.789148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033a610-6570-4bc1-ac65-d98800a0fa7f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:28.806981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.807794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.808187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.808462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.818263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:28.825912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:28.828011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:28.829005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:28.829419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:28.829796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/db8edbaf5787451c9cd851df007d0f35/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:28.832632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] RBAC: Authorizing `identity:create_grant(project_id=638fa7b1962d4ab7bf021ee0225dd847, user_id=db8edbaf5787451c9cd851df007d0f35, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:28.834245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:28.834363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:28.835212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:28.836007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 66/130] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:28.836650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.885090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:28.936000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 65/131] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:28 2022] PUT /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/db8edbaf5787451c9cd851df007d0f35/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:28.936779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:28.993442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1944be6-0e7a-48de-b924-c0027bfff338 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:29.082173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:29.083245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:29.083536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:29.083817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:29.085349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:29.089083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:29.089230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 67/132] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:28 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:29.096152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:29.184853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd992873-d062-44f7-95c3-6fbd56549c22 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:29.255228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:29.255851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:29.256137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:29.256415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:29.257189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] RBAC: Adding query filter params (name=member) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:29.257529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:29.258925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:29.273351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 66/133] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:29.278838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:29.340381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62a3e4-880b-40a9-9fd5-005c82317d1b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:29.361567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:29.369623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:29.398397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:29.399301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:29.399583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:29.399857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/90d9e3351e21469a95b1a88347c81225/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:29.403193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=638fa7b1962d4ab7bf021ee0225dd847, user_id=90d9e3351e21469a95b1a88347c81225, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:29.404678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:29.441905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49b43d8-66e1-4b8f-a849-422a9549a02d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:29.467750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 68/134] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:29 2022] PUT /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/90d9e3351e21469a95b1a88347c81225/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:29.488695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:29.489784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:29.490109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:29.490482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:29.491811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:29.493303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:29.495191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 67/135] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:29 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:29.506543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:29.641236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cead56-f828-4555-be94-93353b44d7e2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:29.690360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6afe09-175c-47e0-9ff1-d1e7c826d8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:29.691604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6afe09-175c-47e0-9ff1-d1e7c826d8ab None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:29.691902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6afe09-175c-47e0-9ff1-d1e7c826d8ab None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:29.692187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6afe09-175c-47e0-9ff1-d1e7c826d8ab None admin] PATH_INFO: `/v3/projects/d4686cf3d9244e29ba9af47e9112864c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:29.698283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6afe09-175c-47e0-9ff1-d1e7c826d8ab None admin] RBAC: Authorizing `identity:get_project(project_id=d4686cf3d9244e29ba9af47e9112864c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:29.699886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6afe09-175c-47e0-9ff1-d1e7c826d8ab None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:29.702192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 69/136] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:29 2022] GET /identity/v3/projects/d4686cf3d9244e29ba9af47e9112864c => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:29.708807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:29.831858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbe19e-03e3-462e-b7e4-1460f26aa664 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:29.853627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:29.882222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c049a46-0fb3-432b-a276-e7cc0f63b96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:29.883154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049a46-0fb3-432b-a276-e7cc0f63b96d None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:29.883435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049a46-0fb3-432b-a276-e7cc0f63b96d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:29.883716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c049a46-0fb3-432b-a276-e7cc0f63b96d None admin] PATH_INFO: `/v3/projects/d4686cf3d9244e29ba9af47e9112864c/users/90d9e3351e21469a95b1a88347c81225/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:29.885837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c049a46-0fb3-432b-a276-e7cc0f63b96d None admin] RBAC: Authorizing `identity:create_grant(project_id=d4686cf3d9244e29ba9af47e9112864c, user_id=90d9e3351e21469a95b1a88347c81225, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:29.887280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c049a46-0fb3-432b-a276-e7cc0f63b96d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:29.964577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 68/137] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:29 2022] PUT /identity/v3/projects/d4686cf3d9244e29ba9af47e9112864c/users/90d9e3351e21469a95b1a88347c81225/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:30.360742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aab890-0dc3-43d5-9a6b-6420f9099faf None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:30.361244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aab890-0dc3-43d5-9a6b-6420f9099faf None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:30.361406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aab890-0dc3-43d5-9a6b-6420f9099faf None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:30.364042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 70/138] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:30.369293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8b8b-3c08-4d4e-98ac-79ab18e2935a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:30.369741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8b8b-3c08-4d4e-98ac-79ab18e2935a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:30.369934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269c8b8b-3c08-4d4e-98ac-79ab18e2935a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:30.388502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-269c8b8b-3c08-4d4e-98ac-79ab18e2935a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:30.390434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-269c8b8b-3c08-4d4e-98ac-79ab18e2935a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:30.467460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 69/139] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:30.493972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aab890-0dc3-43d5-9a6b-6420f9099faf None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:30.556333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:30.646304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aab890-0dc3-43d5-9a6b-6420f9099faf None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:30.692158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c6a68-0680-4226-b467-1dc9889fa6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:30.692968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6a68-0680-4226-b467-1dc9889fa6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:30.693258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6a68-0680-4226-b467-1dc9889fa6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:30.693559 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c6a68-0680-4226-b467-1dc9889fa6a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:30.694757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c6a68-0680-4226-b467-1dc9889fa6a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:30.696219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c6a68-0680-4226-b467-1dc9889fa6a3 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:30.698173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 71/140] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:30.703570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8b8b-3c08-4d4e-98ac-79ab18e2935a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:30.834751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269c8b8b-3c08-4d4e-98ac-79ab18e2935a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:30.843195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:30.877332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99975c26-90af-4eb6-8810-b2530d5db6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:30.879024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99975c26-90af-4eb6-8810-b2530d5db6c2 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:30.879512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99975c26-90af-4eb6-8810-b2530d5db6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:30.879998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99975c26-90af-4eb6-8810-b2530d5db6c2 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:30.880911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99975c26-90af-4eb6-8810-b2530d5db6c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:30.882620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99975c26-90af-4eb6-8810-b2530d5db6c2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:30.970346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 70/141] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:52:30 2022] POST /identity/v3/users => generated 283 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:31.194307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030405ce-896a-4818-b758-34a2b5552bfb None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:31.194626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030405ce-896a-4818-b758-34a2b5552bfb None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:31.194881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030405ce-896a-4818-b758-34a2b5552bfb None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:31.196581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 72/142] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:31.202215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdbe8e-d056-456f-b3c6-1d17e9a50b12 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:31.202527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdbe8e-d056-456f-b3c6-1d17e9a50b12 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:31.209072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbdbe8e-d056-456f-b3c6-1d17e9a50b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:31.232451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-acbdbe8e-d056-456f-b3c6-1d17e9a50b12 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:31.233732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-acbdbe8e-d056-456f-b3c6-1d17e9a50b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:31.324893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 71/143] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:31.366904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030405ce-896a-4818-b758-34a2b5552bfb None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:31.491399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030405ce-896a-4818-b758-34a2b5552bfb None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:31.499121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:31.527215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:31.528049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:31.528358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:31.528641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:31.542795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:31.551053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:31.551998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:31.552903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 73/144] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:31.559583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdbe8e-d056-456f-b3c6-1d17e9a50b12 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:31.674857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:31.689485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbdbe8e-d056-456f-b3c6-1d17e9a50b12 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:31.743640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac7e8d-0b89-4976-935f-6b12b897b5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:31.744260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac7e8d-0b89-4976-935f-6b12b897b5ca None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:31.744547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac7e8d-0b89-4976-935f-6b12b897b5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:31.744827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac7e8d-0b89-4976-935f-6b12b897b5ca None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:31.745686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac7e8d-0b89-4976-935f-6b12b897b5ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:31.745992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac7e8d-0b89-4976-935f-6b12b897b5ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:31.747821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac7e8d-0b89-4976-935f-6b12b897b5ca None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:31.758923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 72/145] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:31.764568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:31.891100 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff532ff-bf74-492c-b2c0-07b344c49978 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:31.907067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da45f5e-c9eb-4793-9fa3-00463efd73a0 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:31.907359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da45f5e-c9eb-4793-9fa3-00463efd73a0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:31.907639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da45f5e-c9eb-4793-9fa3-00463efd73a0 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:31.909121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 73/146] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:31.914444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac90ab-c985-405a-b2d9-959804214287 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:31.914738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac90ab-c985-405a-b2d9-959804214287 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:31.915018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac90ab-c985-405a-b2d9-959804214287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:31.938547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c8ac90ab-c985-405a-b2d9-959804214287 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:31.939889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c8ac90ab-c985-405a-b2d9-959804214287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:31.951533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:31.952675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:31.953108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:31.953672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] PATH_INFO: `/v3/users/fef81a30ac58471b96a6d34f76d4e10d` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:31.959832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] RBAC: Authorizing `identity:get_user(user_id=fef81a30ac58471b96a6d34f76d4e10d)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:31.961553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:31.965244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 74/147] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:31 2022] GET /identity/v3/users/fef81a30ac58471b96a6d34f76d4e10d => generated 289 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:31.982419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:32.050210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:32.058771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 74/148] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:32.102855 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac90ab-c985-405a-b2d9-959804214287 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:32.133981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8717e8-ab4f-4498-be26-b5fe382a3675 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:32.190689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:32.194076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:32.195151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:32.195151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:32.198920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:32.206373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:32.206756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 75/149] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:31 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:32.208868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:32.330021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:32.360391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac90ab-c985-405a-b2d9-959804214287 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:32.434563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f1263-e358-40bb-baeb-43aa6e7ba11e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:32.440365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:32.441196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:32.441503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:32.441937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:32.443349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:32.497808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:32.499114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:32.499591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:32.500070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/fef81a30ac58471b96a6d34f76d4e10d/roles/1268b0be014043318348a2854238939a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:32.506703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] RBAC: Authorizing `identity:create_grant(project_id=638fa7b1962d4ab7bf021ee0225dd847, user_id=fef81a30ac58471b96a6d34f76d4e10d, role_id=1268b0be014043318348a2854238939a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:32.508443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:32.517926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:32.536386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:32.537384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:32.545165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 75/150] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:52:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 437 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:32.545854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:32.614757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 76/151] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:32 2022] PUT /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/users/fef81a30ac58471b96a6d34f76d4e10d/roles/1268b0be014043318348a2854238939a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:32.750091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45e78f-ff43-4f5f-878b-f34457497040 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:32.789746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8fe75-4448-4424-b193-a9dadade8387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:32.790440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8fe75-4448-4424-b193-a9dadade8387 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:32.790733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8fe75-4448-4424-b193-a9dadade8387 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:32.791012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8fe75-4448-4424-b193-a9dadade8387 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:32.791789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8fe75-4448-4424-b193-a9dadade8387 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:32.792100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8fe75-4448-4424-b193-a9dadade8387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:32.793525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8fe75-4448-4424-b193-a9dadade8387 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:32.803692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 76/152] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:52:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:32.809411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:32.880877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8f34d-c46b-4a8d-8254-66c57329a6fe None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:32.881255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8f34d-c46b-4a8d-8254-66c57329a6fe None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:32.881658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f8f34d-c46b-4a8d-8254-66c57329a6fe None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:32.883152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 77/153] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:32.888728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75368ff-1a38-49a9-bbd3-9e89439f203f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:32.889130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75368ff-1a38-49a9-bbd3-9e89439f203f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:32.889532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75368ff-1a38-49a9-bbd3-9e89439f203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:32.914358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:32.927301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d75368ff-1a38-49a9-bbd3-9e89439f203f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:32.928701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d75368ff-1a38-49a9-bbd3-9e89439f203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:32.950440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d5f93-bbee-40d2-ba48-8421a86b6ce1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:32.990278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:32.991423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:32.991855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:32.992287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:32.993682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:32.995449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:33.001754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 77/154] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:32 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:33.004414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:33.045415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:33.088028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 78/155] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:33.126325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbc31fb-cdf3-487d-82be-52aa04f85caa None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:33.141841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75368ff-1a38-49a9-bbd3-9e89439f203f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:33.178387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890753f8-7c89-4f5a-ba25-54dad5b5fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:33.179285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890753f8-7c89-4f5a-ba25-54dad5b5fa94 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.179742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890753f8-7c89-4f5a-ba25-54dad5b5fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.180156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890753f8-7c89-4f5a-ba25-54dad5b5fa94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.261814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890753f8-7c89-4f5a-ba25-54dad5b5fa94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:33.263515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890753f8-7c89-4f5a-ba25-54dad5b5fa94 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:33.295252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:33.371631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 78/156] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:52:33.375520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee72ab-7249-4c35-ad13-2f2c6e221c9b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.375954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee72ab-7249-4c35-ad13-2f2c6e221c9b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.376367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee72ab-7249-4c35-ad13-2f2c6e221c9b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.378673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 79/157] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:33.382441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee72ab-7249-4c35-ad13-2f2c6e221c9b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:33.417213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75368ff-1a38-49a9-bbd3-9e89439f203f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:33.493875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee72ab-7249-4c35-ad13-2f2c6e221c9b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:33.502011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:33.503693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.504171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.504553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.510292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:33.527428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:33.534284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba8196f-9b90-4cc9-a5ab-93a1eacaccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:33.535215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:33.535710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8196f-9b90-4cc9-a5ab-93a1eacaccce None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.536131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8196f-9b90-4cc9-a5ab-93a1eacaccce None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.536552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba8196f-9b90-4cc9-a5ab-93a1eacaccce None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.537923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8196f-9b90-4cc9-a5ab-93a1eacaccce None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:33.539958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba8196f-9b90-4cc9-a5ab-93a1eacaccce None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:33.544720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:33.545615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:33.546386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:33.547115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 80/158] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:33 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:33.548605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2affe-1a83-4711-bc73-278097879d45 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.549018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2affe-1a83-4711-bc73-278097879d45 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.551560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d2affe-1a83-4711-bc73-278097879d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.551800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 79/159] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:33.557403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:33.570935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-85d2affe-1a83-4711-bc73-278097879d45 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:33.572376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-85d2affe-1a83-4711-bc73-278097879d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:33.645880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 81/160] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:33.647128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2affe-1a83-4711-bc73-278097879d45 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:33.752521 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d2affe-1a83-4711-bc73-278097879d45 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:33.781821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:33.791347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:33.792212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.796536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.796536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.796536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] RBAC: Adding query filter params (role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=90d9e3351e21469a95b1a88347c81225) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:33.796536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:33.796536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:33.805394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 82/161] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:33 2022] GET /identity/v3/role_assignments?user.id=90d9e3351e21469a95b1a88347c81225&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:33.806465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:33.821313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16827b0-989e-48c5-8fb5-a3b7cb2dfc79 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:33.900031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:33.900692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.901079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.901488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.902355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] RBAC: Adding query filter params (name=member) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:33.902768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:33.904244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:33.927931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 80/162] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:33.933756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:33.939200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ad444-b0a8-4f95-afa8-ebe89a99f7a0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:33.977434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:33.978475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:33.978893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:33.979331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:33.991549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:33.999716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:34.000633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:34.001682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:34.004602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 83/163] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:34.010406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:34.073814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:34.087022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:34.129287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1cd389-6756-4c0f-a0e0-ffa95dc947fc None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:34.192933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:34.194023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:34.194502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:34.194973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:34.195944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:34.196446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:34.198251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:34.208504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 84/164] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:34.213868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:34.227474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a47632-973f-4f67-8ffc-40b7fa994667 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:34.339095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:34.339913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:34.340196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:34.340472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:34.341749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:34.343218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:34.355186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 81/165] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:33 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:34.355186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:34.356562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8b4fb-4fa6-4df7-b36d-07455e97c736 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:34.376831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:34.431170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:34.432086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:34.432374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:34.432653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] PATH_INFO: `/v3/users/db8edbaf5787451c9cd851df007d0f35` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:34.433965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] RBAC: Authorizing `identity:get_user(user_id=db8edbaf5787451c9cd851df007d0f35)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:34.435456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:34.437258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 85/166] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:34 2022] GET /identity/v3/users/db8edbaf5787451c9cd851df007d0f35 => generated 247 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:34.451022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:34.523993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e971ac8-c16d-4d26-8214-bcf2d8ed7982 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:34.594024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:34.615631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c95c2c-aab9-4633-9d93-987981067382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:34.616295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c95c2c-aab9-4633-9d93-987981067382 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:34.616582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c95c2c-aab9-4633-9d93-987981067382 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:34.616863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c95c2c-aab9-4633-9d93-987981067382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:34.698778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171ceaa-9c29-443c-beb5-ab1b2505807b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:34.769571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c95c2c-aab9-4633-9d93-987981067382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:34.771074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c95c2c-aab9-4633-9d93-987981067382 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:34.775660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:34.776325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:34.776607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:34.776881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:34.876128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:34.907402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:34.909057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:34.965354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 82/167] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:34.966706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6117e2-d57b-4404-bf47-bd015c7c169b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:34.966994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6117e2-d57b-4404-bf47-bd015c7c169b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:34.967342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6117e2-d57b-4404-bf47-bd015c7c169b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:34.968635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 83/168] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:34.974892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cba00-b035-4101-abaa-5cb932870135 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:34.975179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cba00-b035-4101-abaa-5cb932870135 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:34.975459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2cba00-b035-4101-abaa-5cb932870135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:35.006283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-dc2cba00-b035-4101-abaa-5cb932870135 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:35.007483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-dc2cba00-b035-4101-abaa-5cb932870135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:35.033157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:35.095877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 86/169] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:52:35.097652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:35.123944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 84/170] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:35.124816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cba00-b035-4101-abaa-5cb932870135 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:35.289007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2cba00-b035-4101-abaa-5cb932870135 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:35.301775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:35.331745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1ca9b-3193-4d10-80a2-b1fd2d301eb4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:35.334916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:35.335805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:35.336085 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:35.336371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:35.337602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:35.339060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:35.340757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 85/171] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:35 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:35.341550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:35.390841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:35.391713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:35.391994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:35.392270 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:35.393686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:35.398548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:35.400055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:35.401852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 87/172] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:35 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:35.402624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:35.540906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff860bfa-8bbe-43f8-bb2c-ec46a16df908 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:35.597646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:35.598450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:35.598734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:35.599014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:35.609512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:35.613783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbb24b-5666-43ac-a8e8-f05accc7e170 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:35.617899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:35.618756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:35.619543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 86/173] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:35.620207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:35.678431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:35.679104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:35.679451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:35.679727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:35.680578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=db8edbaf5787451c9cd851df007d0f35) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:35.680887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:35.682370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:35.694573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 88/174] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:35 2022] GET /identity/v3/role_assignments?user.id=db8edbaf5787451c9cd851df007d0f35&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:35.695462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:35.730376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:35.732470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:35.761527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0a052-cee0-4c50-8eb6-a5f6be248919 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:35.840798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:35.841423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:35.841707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:35.841987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:35.842842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=90d9e3351e21469a95b1a88347c81225) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:35.843156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:35.844630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:35.851911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 87/175] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:35 2022] GET /identity/v3/role_assignments?user.id=90d9e3351e21469a95b1a88347c81225&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:35.879792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b37d7-15b8-4d59-a381-fe9179ab89b7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:35.919196 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86a8cb0-44b3-450b-babd-71487b48186c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:35.919962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8cb0-44b3-450b-babd-71487b48186c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:35.920256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8cb0-44b3-450b-babd-71487b48186c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:35.920538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86a8cb0-44b3-450b-babd-71487b48186c None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:35.921388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8cb0-44b3-450b-babd-71487b48186c None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:35.921703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8cb0-44b3-450b-babd-71487b48186c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:35.923167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86a8cb0-44b3-450b-babd-71487b48186c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:35.933182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 89/176] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:35.942444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.047069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb66af-ef9c-448e-8509-31fcae12e0d5 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.047394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb66af-ef9c-448e-8509-31fcae12e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.047614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb66af-ef9c-448e-8509-31fcae12e0d5 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.049052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 90/177] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:36.059685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed21c5-ef64-4d18-ac32-118baa276a35 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.060301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed21c5-ef64-4d18-ac32-118baa276a35 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.060803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed21c5-ef64-4d18-ac32-118baa276a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.078564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828adb4-be6f-49cb-ac69-2e73a49319af None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:36.102320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-d7ed21c5-ef64-4d18-ac32-118baa276a35 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:36.103471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d7ed21c5-ef64-4d18-ac32-118baa276a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:36.115191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:36.116098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.116382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.116658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] PATH_INFO: `/v3/users/90d9e3351e21469a95b1a88347c81225` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.117931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] RBAC: Authorizing `identity:get_user(user_id=90d9e3351e21469a95b1a88347c81225)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:36.119397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:36.121040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 88/178] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:35 2022] GET /identity/v3/users/90d9e3351e21469a95b1a88347c81225 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:36.130707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.169003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:36.259895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 91/179] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:36.313633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed21c5-ef64-4d18-ac32-118baa276a35 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.324727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c2493-fde3-469b-af1f-7b916867e82a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:36.333744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:36.380670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:36.381261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.381567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.381848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.464855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed21c5-ef64-4d18-ac32-118baa276a35 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:36.469033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:36.470731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:36.495969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:36.523997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d523de62-f854-4424-9ba6-7bc913360de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:36.524879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523de62-f854-4424-9ba6-7bc913360de7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.525166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523de62-f854-4424-9ba6-7bc913360de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.525473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d523de62-f854-4424-9ba6-7bc913360de7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.547498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d523de62-f854-4424-9ba6-7bc913360de7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-d523de62-f854-4424-9ba6-7bc913360de7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:36.558857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:36.559965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:36.560892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:36.563366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 92/180] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:36.564903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a3b1f-50ab-412f-84da-d996f1085220 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.565904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a3b1f-50ab-412f-84da-d996f1085220 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.566377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a3b1f-50ab-412f-84da-d996f1085220 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.567861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 93/181] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:36.574714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e38b9-6153-42ca-870d-5aa6437602e0 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.575001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e38b9-6153-42ca-870d-5aa6437602e0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.575278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e38b9-6153-42ca-870d-5aa6437602e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.592425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-420e38b9-6153-42ca-870d-5aa6437602e0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:36.593718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-420e38b9-6153-42ca-870d-5aa6437602e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:36.606831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 89/182] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:52:36.607947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.670716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:36.681050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 94/183] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:36.687070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e38b9-6153-42ca-870d-5aa6437602e0 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.737710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8453c4-f80e-4b57-94b1-7f40f01c6351 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:36.753845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:36.771912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:36.773011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.773333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.773617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.774759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:36.775075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:36.776804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:36.790877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 90/184] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:36.791757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.800449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e38b9-6153-42ca-870d-5aa6437602e0 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:36.837027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42723b6f-786b-4528-8905-1e6490ee1e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:36.837915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723b6f-786b-4528-8905-1e6490ee1e81 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.838195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723b6f-786b-4528-8905-1e6490ee1e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.838465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42723b6f-786b-4528-8905-1e6490ee1e81 None admin] PATH_INFO: `/v3/projects/d4686cf3d9244e29ba9af47e9112864c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.839775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42723b6f-786b-4528-8905-1e6490ee1e81 None admin] RBAC: Authorizing `identity:get_project(project_id=d4686cf3d9244e29ba9af47e9112864c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:36.841640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42723b6f-786b-4528-8905-1e6490ee1e81 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:36.843334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 95/185] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:36 2022] GET /identity/v3/projects/d4686cf3d9244e29ba9af47e9112864c => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:36.849999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8ff62-acd7-495c-b049-fd2d2ceb4c5d None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.850289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8ff62-acd7-495c-b049-fd2d2ceb4c5d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.850571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8ff62-acd7-495c-b049-fd2d2ceb4c5d None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.852136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 96/186] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:36.852718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8ff62-acd7-495c-b049-fd2d2ceb4c5d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.909382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c13c-aa3b-48df-a68b-d981eb1aebdc None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:36.923434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:36.945707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:36.946799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:36.947244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:36.947523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:36.948706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:36.950191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:36.952001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 91/187] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:36.952566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:36.970696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8ff62-acd7-495c-b049-fd2d2ceb4c5d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.004998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:37.013369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafdb792-48c0-47d0-b747-ddff6e64d21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.014103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdb792-48c0-47d0-b747-ddff6e64d21e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.018870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdb792-48c0-47d0-b747-ddff6e64d21e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.018870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafdb792-48c0-47d0-b747-ddff6e64d21e None admin] PATH_INFO: `/v3/users/35572f1c637c4421959b0ba9c478df96` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.030704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdb792-48c0-47d0-b747-ddff6e64d21e None admin] RBAC: Authorizing `identity:get_user(user_id=35572f1c637c4421959b0ba9c478df96)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:37.032456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafdb792-48c0-47d0-b747-ddff6e64d21e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:37.035456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 97/188] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:36 2022] GET /identity/v3/users/35572f1c637c4421959b0ba9c478df96 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:37.040964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced4653-cc46-4db2-a956-65c5acf3275b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.041260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced4653-cc46-4db2-a956-65c5acf3275b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.041587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced4653-cc46-4db2-a956-65c5acf3275b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.059694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1ced4653-cc46-4db2-a956-65c5acf3275b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:37.060876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1ced4653-cc46-4db2-a956-65c5acf3275b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:37.062670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e0969-2da3-4130-af21-d34b12861f6e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.097652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.098312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.098585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.098862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.099692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=d4686cf3d9244e29ba9af47e9112864c, user.id=90d9e3351e21469a95b1a88347c81225) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:37.100004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:37.101462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:37.107204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 92/189] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:36 2022] GET /identity/v3/role_assignments?user.id=90d9e3351e21469a95b1a88347c81225&scope.project.id=d4686cf3d9244e29ba9af47e9112864c&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:37.107961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.134610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 98/190] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:37.139238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced4653-cc46-4db2-a956-65c5acf3275b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.210896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:37.227115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd1a12-27aa-4b2a-8c32-859f888f9596 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.239213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:37.263511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced4653-cc46-4db2-a956-65c5acf3275b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.268163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c8453-ed2f-40b8-835f-05b624327384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.269047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8453-ed2f-40b8-835f-05b624327384 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.269420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8453-ed2f-40b8-835f-05b624327384 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.269635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c8453-ed2f-40b8-835f-05b624327384 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.270385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c8453-ed2f-40b8-835f-05b624327384 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:37.271919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c8453-ed2f-40b8-835f-05b624327384 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:37.304568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81cb42-2d46-4e80-a7f5-c6bd84f45d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.305358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81cb42-2d46-4e80-a7f5-c6bd84f45d26 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.305647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81cb42-2d46-4e80-a7f5-c6bd84f45d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.305922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81cb42-2d46-4e80-a7f5-c6bd84f45d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.370483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 93/191] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:52:37 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:37.371964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3c756-204f-48e7-97f9-cd9e20ca3561 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.372368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3c756-204f-48e7-97f9-cd9e20ca3561 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.372540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3c756-204f-48e7-97f9-cd9e20ca3561 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.373942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 94/192] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:37.374710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3c756-204f-48e7-97f9-cd9e20ca3561 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.381692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81cb42-2d46-4e80-a7f5-c6bd84f45d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:37.383254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81cb42-2d46-4e80-a7f5-c6bd84f45d26 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:37.455807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:37.492851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3c756-204f-48e7-97f9-cd9e20ca3561 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.526004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 99/193] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:37.528208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f577-af1f-4a51-b9bb-96d724b38d0a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.528631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f577-af1f-4a51-b9bb-96d724b38d0a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.529034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f577-af1f-4a51-b9bb-96d724b38d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.542648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.543908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.544366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.544807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.550191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-dc54f577-af1f-4a51-b9bb-96d724b38d0a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:37.551780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-dc54f577-af1f-4a51-b9bb-96d724b38d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:37.562509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:37.576930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:37.578088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:37.579098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 95/194] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:52:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:37.580315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.628463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:37.652848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 100/195] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:37.654285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54f577-af1f-4a51-b9bb-96d724b38d0a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.735492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af130c-0bb5-4085-becc-f95c3fa90959 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.766888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:37.779555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.780427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.783026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.783026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.783026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:37.783889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:37.786959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 96/196] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:37 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:37.786959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.794992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54f577-af1f-4a51-b9bb-96d724b38d0a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.843994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.844742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.845004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.845349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.846633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:37.846998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:37.849186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:37.860887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 101/197] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:52:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:37.862711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.902679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:37.911614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ea420-6efd-49a7-8695-ecd0934d65ea None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:37.953331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:37.954444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:37.954935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:37.955390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:37.967841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:37.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:37.978682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:37.979525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:37.982056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 97/198] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:37.987688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:37.993118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536de186-8313-4feb-bd8f-1b54e5705952 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:38.039258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:38.039849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:38.040133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:38.040407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:38.041249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=35572f1c637c4421959b0ba9c478df96) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:38.041770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:38.043231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:38.049126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 102/199] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:37 2022] GET /identity/v3/role_assignments?user.id=35572f1c637c4421959b0ba9c478df96&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:38.050137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:38.062598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:38.138476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b3098-03dc-44c4-a7c3-da042d0ffb2e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:38.177405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51eef1-1396-4ee8-88e6-7af82bb42260 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:38.208346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:38.209280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:38.209568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:38.209837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] PATH_INFO: `/v3/users/35572f1c637c4421959b0ba9c478df96` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:38.211047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] RBAC: Authorizing `identity:get_user(user_id=35572f1c637c4421959b0ba9c478df96)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:38.218548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:38.221204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 98/200] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:37 2022] GET /identity/v3/users/35572f1c637c4421959b0ba9c478df96 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:38.224058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:38.225073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:38.225692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:38.226231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:38.227387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:38.231413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:38.232062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:38.234849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:38.255527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 103/201] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:38.262294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:38.288564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:38.352849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:38.388856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac46ff-dd42-414f-a564-2984c85bcb98 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:38.407370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa900a7-8e8c-4b17-955a-1914d1aced9f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:38.439461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:38.440981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:38.443985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:38.444480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] PATH_INFO: `/v3/users/fef81a30ac58471b96a6d34f76d4e10d` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:38.450189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] RBAC: Authorizing `identity:get_user(user_id=fef81a30ac58471b96a6d34f76d4e10d)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:38.451914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:38.455250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 104/202] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:38 2022] GET /identity/v3/users/fef81a30ac58471b96a6d34f76d4e10d => generated 289 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:38.463016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:38.491955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:38.492990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:38.493502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:38.493982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:38.641725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:38.643109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:38.649388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a4b60-ace6-40d1-8159-44d62dca3a3a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:38.692194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:38.737286 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:38.738093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:38.738377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:38.738655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:38.762016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:38.794492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 99/203] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:38.800766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:38.885685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:38.887173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:38.989712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbdcaf-c5a0-42c2-83c3-48e005fed6cf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:39.028028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:39.052458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:39.053585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:39.055492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:39.055802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:39.058824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 105/204] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:39.059586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:39.061340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:39.063183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 100/205] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:38 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:39.069111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:39.072741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:39.138582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:39.265778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44068fed-e944-4869-b59e-b95e35223c34 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:39.282617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72414e99-944e-4338-8a7f-bd2f67ce5abe None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:39.339178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:39.340060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:39.340341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:39.340623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:39.343729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:39.345203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:39.347227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 106/206] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:39 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:39.351643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab9aad-24ce-4419-a462-854d1d70e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:39.352406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9aad-24ce-4419-a462-854d1d70e87c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:39.352813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9aad-24ce-4419-a462-854d1d70e87c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:39.353019 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab9aad-24ce-4419-a462-854d1d70e87c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:39.353899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9aad-24ce-4419-a462-854d1d70e87c None admin] RBAC: Adding query filter params (role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=35572f1c637c4421959b0ba9c478df96) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:39.354217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9aad-24ce-4419-a462-854d1d70e87c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:39.355659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab9aad-24ce-4419-a462-854d1d70e87c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:39.358301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:39.371347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 101/207] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:39 2022] GET /identity/v3/role_assignments?user.id=35572f1c637c4421959b0ba9c478df96&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 259 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:39.499349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d962c6-6fe6-41d1-8045-a69d5b186c00 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:39.508876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:39.553223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1895a4-5a6b-4a38-a26f-4ec9a0f44bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:39.553860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1895a4-5a6b-4a38-a26f-4ec9a0f44bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:39.554146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1895a4-5a6b-4a38-a26f-4ec9a0f44bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:39.554423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1895a4-5a6b-4a38-a26f-4ec9a0f44bd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:39.555697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1895a4-5a6b-4a38-a26f-4ec9a0f44bd9 None admin] RBAC: Adding query filter params (role.id=1268b0be014043318348a2854238939a, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847, user.id=fef81a30ac58471b96a6d34f76d4e10d) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:39.556076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1895a4-5a6b-4a38-a26f-4ec9a0f44bd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:39.557639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1895a4-5a6b-4a38-a26f-4ec9a0f44bd9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:39.570469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 107/208] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:39 2022] GET /identity/v3/role_assignments?user.id=fef81a30ac58471b96a6d34f76d4e10d&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=1268b0be014043318348a2854238939a => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:39.886308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779bc9d-14b6-4f7a-a8c6-daff938af6f5 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:39.886647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779bc9d-14b6-4f7a-a8c6-daff938af6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:39.886800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779bc9d-14b6-4f7a-a8c6-daff938af6f5 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:39.888476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 102/209] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:39.894482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590ab56-1dc6-40e2-805e-6e382252a063 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:39.894946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590ab56-1dc6-40e2-805e-6e382252a063 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:39.895120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590ab56-1dc6-40e2-805e-6e382252a063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:39.916308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f590ab56-1dc6-40e2-805e-6e382252a063 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:39.917527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f590ab56-1dc6-40e2-805e-6e382252a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:39.994314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 108/210] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:40.017805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779bc9d-14b6-4f7a-a8c6-daff938af6f5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:40.051841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f27e9d-6fe8-4290-9f04-e490f3106dde None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.052146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f27e9d-6fe8-4290-9f04-e490f3106dde None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.052402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f27e9d-6fe8-4290-9f04-e490f3106dde None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.054110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 109/211] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:40.059683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d314b-ab48-4254-b763-c7eb2e670e1b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.059964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d314b-ab48-4254-b763-c7eb2e670e1b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.060239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d314b-ab48-4254-b763-c7eb2e670e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.083263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fa5d314b-ab48-4254-b763-c7eb2e670e1b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:40.084896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fa5d314b-ab48-4254-b763-c7eb2e670e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:40.087596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:40.155952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779bc9d-14b6-4f7a-a8c6-daff938af6f5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:40.186049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 110/212] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:40.201898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:40.203699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.203699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.203699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.215365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:40.218451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d314b-ab48-4254-b763-c7eb2e670e1b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:40.224991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:40.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:40.227948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 103/213] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:40.232460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:40.260026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:40.357176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d314b-ab48-4254-b763-c7eb2e670e1b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:40.360698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eef5a9-f942-4dc8-a377-254ebde3f813 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:40.376944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:40.400366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:40.401234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.401643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.402069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.402884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:40.403126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:40.403536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:40.403684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.403975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.404547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.405343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:40.405766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:40.407474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:40.413375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 111/214] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:40.415842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:40.424803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 104/215] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:40.426626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:40.539871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45da489-4de7-45bd-96e9-5d74192fddbf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:40.551958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:40.557302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f7998-4b30-438d-94b3-c944f8d638c1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:40.577309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:40.578359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.578660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.578926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] PATH_INFO: `/v3/users/353141c307974b078a700bb724209420` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.597419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] RBAC: Authorizing `identity:get_user(user_id=353141c307974b078a700bb724209420)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:40.599032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:40.602084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c6a45-dafc-4b38-8e18-69d241d26f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:40.602956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c6a45-dafc-4b38-8e18-69d241d26f8b None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.603505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c6a45-dafc-4b38-8e18-69d241d26f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.603942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c6a45-dafc-4b38-8e18-69d241d26f8b None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.604817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c6a45-dafc-4b38-8e18-69d241d26f8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:40.605511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 105/216] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:40 2022] GET /identity/v3/users/353141c307974b078a700bb724209420 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:40.606848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:40.607134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c6a45-dafc-4b38-8e18-69d241d26f8b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:40.651068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:40.701324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 112/217] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:52:40 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:52:40.740648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b813b-8781-4e11-9c15-225e2c1ce3a1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:40.780783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:40.781494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.781783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.782064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.826260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41acef6-c67b-4cd8-affd-df331cec8159 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.826589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41acef6-c67b-4cd8-affd-df331cec8159 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.826809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41acef6-c67b-4cd8-affd-df331cec8159 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.828482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 113/218] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:40.835057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222fb11-b512-46b4-8a7d-dd7925e136d5 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:40.835356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222fb11-b512-46b4-8a7d-dd7925e136d5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:40.835620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e222fb11-b512-46b4-8a7d-dd7925e136d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:40.854201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e222fb11-b512-46b4-8a7d-dd7925e136d5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:40.855404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e222fb11-b512-46b4-8a7d-dd7925e136d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:40.867024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:40.869004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:40.908607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:40.933232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 114/219] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:40.955576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222fb11-b512-46b4-8a7d-dd7925e136d5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:40.993972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 106/220] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:40.999420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:41.065102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:41.090093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e222fb11-b512-46b4-8a7d-dd7925e136d5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:41.122387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9027983-75d1-4ba7-b25b-4540a338d215 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:41.158772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:41.162431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:41.162431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:41.162431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:41.168010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:41.169089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:41.169571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:41.170008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:41.171500 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:41.173123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:41.175930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 107/221] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:40 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:41.182174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:41.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:41.190904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:41.191706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 115/222] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:52:41.195851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:41.196734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:41.200051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:41.414638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ba2b0-12f3-4341-bcf5-821026dbd30a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:41.459552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:41.460243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:41.460522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:41.460793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:41.461575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:41.461882 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:41.463281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:41.478792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3129fdb-7628-440c-94ec-38dacfad7447 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:41.485453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 116/223] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:41.497017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:41.518878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:41.528344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:41.529009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:41.529310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:41.529595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:41.530443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=353141c307974b078a700bb724209420) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:41.530757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:41.532218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:41.548706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 108/224] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:41 2022] GET /identity/v3/role_assignments?user.id=353141c307974b078a700bb724209420&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 259 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:41.655354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e21e3b-469b-40dd-99dc-db1b1b285e59 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:41.703436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:41.704326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:41.704606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:41.704877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] PATH_INFO: `/v3/users/35572f1c637c4421959b0ba9c478df96` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:41.706693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] RBAC: Authorizing `identity:get_user(user_id=35572f1c637c4421959b0ba9c478df96)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:41.708264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:41.710114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 117/225] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:41 2022] GET /identity/v3/users/35572f1c637c4421959b0ba9c478df96 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:41.715958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:41.795236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:41.833354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c291ef-6883-4549-846d-40dbc6c21ee1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:41.869575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad884c79-3126-46ae-a601-8e995ebcb04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:41.870907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884c79-3126-46ae-a601-8e995ebcb04f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:41.871198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884c79-3126-46ae-a601-8e995ebcb04f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:41.871485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad884c79-3126-46ae-a601-8e995ebcb04f None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:41.873067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884c79-3126-46ae-a601-8e995ebcb04f None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:41.874768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad884c79-3126-46ae-a601-8e995ebcb04f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:41.877299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 109/226] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:41 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:41.884901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:41.965228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:41.990877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d3c5b-789f-4e8b-acc4-6cc2f39bf585 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:42.024948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:42.025883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:42.026172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:42.026455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119/users/35572f1c637c4421959b0ba9c478df96/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:42.028433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] RBAC: Authorizing `identity:create_grant(project_id=db7757ba61ab423fa983d6fa71116119, user_id=35572f1c637c4421959b0ba9c478df96, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:42.029946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:42.061868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ea16-eda0-4fa3-8851-712297f81fbd None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:42.062164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ea16-eda0-4fa3-8851-712297f81fbd None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:42.062429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ea16-eda0-4fa3-8851-712297f81fbd None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:42.063803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 110/227] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:42.069647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21e52-8668-4168-a5ac-e2cc92c10482 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:42.069935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21e52-8668-4168-a5ac-e2cc92c10482 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:42.070231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f21e52-8668-4168-a5ac-e2cc92c10482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:42.087862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-35f21e52-8668-4168-a5ac-e2cc92c10482 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:42.089079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-35f21e52-8668-4168-a5ac-e2cc92c10482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:42.110846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 118/228] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:41 2022] PUT /identity/v3/projects/db7757ba61ab423fa983d6fa71116119/users/35572f1c637c4421959b0ba9c478df96/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:42.202997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 111/229] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:42.241329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:42.388118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4b104-8962-4022-849b-108483e833ab None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:42.441743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:42.442571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:42.442852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:42.443129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:42.445075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:42.462285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:42.470566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:42.472505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:42.473780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 119/230] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:52:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:42.478293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f21e52-8668-4168-a5ac-e2cc92c10482 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:42.551321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:42.591424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f21e52-8668-4168-a5ac-e2cc92c10482 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:42.629597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:42.630633 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:42.631117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:42.631592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:42.632544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:42.633072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:42.641290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:42.651127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 112/231] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:52:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:42.656405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:42.824132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f0e50-b478-43a0-953e-ebe12fb8405e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:42.871315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:42.872198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:42.872476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:42.872747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] PATH_INFO: `/v3/users/35572f1c637c4421959b0ba9c478df96` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:42.873965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] RBAC: Authorizing `identity:get_user(user_id=35572f1c637c4421959b0ba9c478df96)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:42.875429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:42.877374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 120/232] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:42 2022] GET /identity/v3/users/35572f1c637c4421959b0ba9c478df96 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:42.882770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:43.036311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a70179-8ebd-4b2f-92dd-283cebc3a79d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:43.069837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:43.076764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d3c22-f898-455a-8e46-31fb7e1778cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:43.077604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3c22-f898-455a-8e46-31fb7e1778cb None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:43.077887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3c22-f898-455a-8e46-31fb7e1778cb None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:43.078162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d3c22-f898-455a-8e46-31fb7e1778cb None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:43.082108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d3c22-f898-455a-8e46-31fb7e1778cb None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:43.095218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d3c22-f898-455a-8e46-31fb7e1778cb None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:43.095218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 113/233] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:42 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:43.095401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:43.170216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:43.210012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b20ab8f-3665-4dc0-81ef-222ea8a1b734 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:43.253281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ba491-8be8-4c5f-918d-6d5fa5c1116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:43.254241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ba491-8be8-4c5f-918d-6d5fa5c1116f None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:43.254523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ba491-8be8-4c5f-918d-6d5fa5c1116f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:43.254796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ba491-8be8-4c5f-918d-6d5fa5c1116f None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119/users/35572f1c637c4421959b0ba9c478df96/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:43.263589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ba491-8be8-4c5f-918d-6d5fa5c1116f None admin] RBAC: Authorizing `identity:create_grant(project_id=db7757ba61ab423fa983d6fa71116119, user_id=35572f1c637c4421959b0ba9c478df96, role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:43.265110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ba491-8be8-4c5f-918d-6d5fa5c1116f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:43.336373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 121/234] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:43 2022] PUT /identity/v3/projects/db7757ba61ab423fa983d6fa71116119/users/35572f1c637c4421959b0ba9c478df96/roles/60347b7e855240eb9c04a1cb202a3500 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:43.561410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adf93a-f0ef-473c-ad92-7010ba6e4d9d None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:43.562426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adf93a-f0ef-473c-ad92-7010ba6e4d9d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:43.562946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35adf93a-f0ef-473c-ad92-7010ba6e4d9d None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:43.564554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 114/235] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:43.571050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d6b74-35d1-4fa6-ac04-fedff1339c71 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:43.571367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d6b74-35d1-4fa6-ac04-fedff1339c71 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:43.571622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d6b74-35d1-4fa6-ac04-fedff1339c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:43.598555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4b3d6b74-35d1-4fa6-ac04-fedff1339c71 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:43.599690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4b3d6b74-35d1-4fa6-ac04-fedff1339c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:43.686736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 122/236] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:43.699703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e2576-1add-4389-b02a-92834d4f36bd None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:43.699995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e2576-1add-4389-b02a-92834d4f36bd None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:43.700278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281e2576-1add-4389-b02a-92834d4f36bd None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:43.703135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 115/237] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:43.707644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad24b2-23fa-41bd-9823-80a614ea1909 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:43.707934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad24b2-23fa-41bd-9823-80a614ea1909 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:43.708215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad24b2-23fa-41bd-9823-80a614ea1909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:43.729949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e2576-1add-4389-b02a-92834d4f36bd None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:43.743260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-71ad24b2-23fa-41bd-9823-80a614ea1909 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:43.744463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-71ad24b2-23fa-41bd-9823-80a614ea1909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:43.845669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:43.879587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281e2576-1add-4389-b02a-92834d4f36bd None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:43.897752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 123/238] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:43.944533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:43.945419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:43.945710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:43.946042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:43.947322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:43.948869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:43.955194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 116/239] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:43.957473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad24b2-23fa-41bd-9823-80a614ea1909 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:43.957592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:44.081792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:44.133727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32e553-c0cf-410a-ba3f-343ebcc49690 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:44.175871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad24b2-23fa-41bd-9823-80a614ea1909 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:44.182866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:44.183789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.184079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.184354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.211768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:44.219594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:44.220474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:44.221331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 117/240] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:44.227605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:44.235414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f625de-0c4b-4b47-b1c0-c86a27eed0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:44.236113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f625de-0c4b-4b47-b1c0-c86a27eed0b5 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.236399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f625de-0c4b-4b47-b1c0-c86a27eed0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.236681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f625de-0c4b-4b47-b1c0-c86a27eed0b5 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.237673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f625de-0c4b-4b47-b1c0-c86a27eed0b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:44.239350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f625de-0c4b-4b47-b1c0-c86a27eed0b5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:44.310388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 124/241] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:52:43 2022] POST /identity/v3/users => generated 293 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:44.311555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6d043-1cb1-430e-85a2-c6468bcaa7ed None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.311843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6d043-1cb1-430e-85a2-c6468bcaa7ed None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.312149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6d043-1cb1-430e-85a2-c6468bcaa7ed None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.313434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 125/242] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:44.320624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60388d-2079-43c6-aec6-1462186611f6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.321052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60388d-2079-43c6-aec6-1462186611f6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.321458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60388d-2079-43c6-aec6-1462186611f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.340019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2d60388d-2079-43c6-aec6-1462186611f6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:44.341428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2d60388d-2079-43c6-aec6-1462186611f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:44.365902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea4b41-9161-42af-9299-0614332ea6bd None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:44.379413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:44.399015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:44.412280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:44.413109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.413539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.413934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.414813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:44.415223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:44.416741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:44.427644 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 118/243] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:44.434442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 126/244] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:44.435486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:44.498836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60388d-2079-43c6-aec6-1462186611f6 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:44.615321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13c9f3-2a02-4741-ad91-0d796adba005 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:44.635278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60388d-2079-43c6-aec6-1462186611f6 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:44.655168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:44.656013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.656603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.656603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] PATH_INFO: `/v3/users/673c8c67167348cebefb65699ea0f0dc` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.673779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] RBAC: Authorizing `identity:get_user(user_id=673c8c67167348cebefb65699ea0f0dc)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:44.676122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:44.678513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 119/245] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:44 2022] GET /identity/v3/users/673c8c67167348cebefb65699ea0f0dc => generated 297 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:44.683992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:44.687070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:44.689096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.689631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.690258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.704773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:44.715603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:44.716494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:44.717445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:44.719985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 127/246] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:44.726962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:44.747030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:44.766041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:44.823601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f870b-c64c-43f5-bd1f-bc2ddbb9267e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:44.866775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa44b8c-42fa-4d34-88fc-bfb85fc30622 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:44.873128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:44.873918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.874277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.874646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.926097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:44.926821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:44.927113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:44.927393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:44.928363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:44.929571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:44.931015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:44.943442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 128/247] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:44.949033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:44.979600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:44.981196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:45.028615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:45.057767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:45.069940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b66f193-3f10-4672-ade1-5fddf9259304 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:45.092916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 120/248] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:52:45.098773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:45.108228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:45.109242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.109526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.109804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] PATH_INFO: `/v3/users/353141c307974b078a700bb724209420` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.111149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] RBAC: Authorizing `identity:get_user(user_id=353141c307974b078a700bb724209420)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:45.113198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:45.115323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 129/249] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:44 2022] GET /identity/v3/users/353141c307974b078a700bb724209420 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:45.122339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:45.230597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90ba34-d115-4b6c-9bdb-4813a292c32f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:45.247696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cd07a2-5585-4045-a9cf-70455ca78336 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:45.272163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b5dde-7976-45a1-b9e0-7308933de3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:45.273804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b5dde-7976-45a1-b9e0-7308933de3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.274318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b5dde-7976-45a1-b9e0-7308933de3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.274784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b5dde-7976-45a1-b9e0-7308933de3c8 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.276194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b5dde-7976-45a1-b9e0-7308933de3c8 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:45.277920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b5dde-7976-45a1-b9e0-7308933de3c8 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:45.280625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 121/250] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:45 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:45.282639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdf5b-944b-4f06-b72d-9a731621efa7 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.283132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdf5b-944b-4f06-b72d-9a731621efa7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.283589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfdf5b-944b-4f06-b72d-9a731621efa7 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.285336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 122/251] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:45.289613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfdf5b-944b-4f06-b72d-9a731621efa7 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:45.304645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:45.331316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e102f-ca9e-47f2-9974-7e063c85c2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:45.332266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e102f-ca9e-47f2-9974-7e063c85c2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.332547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e102f-ca9e-47f2-9974-7e063c85c2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.332818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e102f-ca9e-47f2-9974-7e063c85c2b6 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.334699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e102f-ca9e-47f2-9974-7e063c85c2b6 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:45.336931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e102f-ca9e-47f2-9974-7e063c85c2b6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:45.338885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 130/252] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:45 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:45.339818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f3638-47cf-4a09-80ee-2703b57d39d8 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.340157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f3638-47cf-4a09-80ee-2703b57d39d8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.340365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f3638-47cf-4a09-80ee-2703b57d39d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.373861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b69f3638-47cf-4a09-80ee-2703b57d39d8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:45.375044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b69f3638-47cf-4a09-80ee-2703b57d39d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:45.414058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:45.479033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfdf5b-944b-4f06-b72d-9a731621efa7 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:45.500095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 131/253] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:45.500842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f3638-47cf-4a09-80ee-2703b57d39d8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:45.545290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:45.545950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.546230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.546505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.547344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] RBAC: Adding query filter params (role.id=1268b0be014043318348a2854238939a, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=673c8c67167348cebefb65699ea0f0dc) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:45.547649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:45.549097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:45.559173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 123/254] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:45 2022] GET /identity/v3/role_assignments?user.id=673c8c67167348cebefb65699ea0f0dc&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=1268b0be014043318348a2854238939a => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:45.559998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:45.654746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f3638-47cf-4a09-80ee-2703b57d39d8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:45.696926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:45.702055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:45.703157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.703500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.703832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119/users/353141c307974b078a700bb724209420/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.710751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7757ba61ab423fa983d6fa71116119, user_id=353141c307974b078a700bb724209420, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:45.712327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:45.726068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56641685-cd72-4954-b88c-bfa62bf6d89a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:45.765984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:45.767118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:45.767541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:45.767955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:45.780894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:45.789441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:45.790581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:52:45.791563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:45.793973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 124/255] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:52:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:45.794360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 132/256] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:45 2022] PUT /identity/v3/projects/db7757ba61ab423fa983d6fa71116119/users/353141c307974b078a700bb724209420/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:45.802457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:45.960546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae6b9a-6e78-4043-a0e1-6652d0a3ab8a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:46.008732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef815739-0a01-477b-94a3-750da22a8c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:46.009447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef815739-0a01-477b-94a3-750da22a8c0a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.009735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef815739-0a01-477b-94a3-750da22a8c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.010015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef815739-0a01-477b-94a3-750da22a8c0a None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.010785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef815739-0a01-477b-94a3-750da22a8c0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:46.011097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef815739-0a01-477b-94a3-750da22a8c0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:46.012536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef815739-0a01-477b-94a3-750da22a8c0a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:46.014182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:46.028173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 125/257] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:46.032513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:46.061213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:46.067348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35ffbb-29e5-482e-a913-70bd1266f54b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.067633 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35ffbb-29e5-482e-a913-70bd1266f54b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.067912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35ffbb-29e5-482e-a913-70bd1266f54b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.069493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 126/258] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:46.075096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06da79-74f9-4494-91ca-98e5c85b04aa None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.075390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06da79-74f9-4494-91ca-98e5c85b04aa None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.075655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06da79-74f9-4494-91ca-98e5c85b04aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.094583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2c06da79-74f9-4494-91ca-98e5c85b04aa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:46.095792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2c06da79-74f9-4494-91ca-98e5c85b04aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:46.217874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa25899-31cb-48b9-80eb-1f952b55b087 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:46.249179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 127/259] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:46.265003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:46.266813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.267428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.267916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] PATH_INFO: `/v3/users/35572f1c637c4421959b0ba9c478df96` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.270131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] RBAC: Authorizing `identity:get_user(user_id=35572f1c637c4421959b0ba9c478df96)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:46.272007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:46.274234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 133/260] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:46 2022] GET /identity/v3/users/35572f1c637c4421959b0ba9c478df96 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:46.282515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06da79-74f9-4494-91ca-98e5c85b04aa None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:46.298087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:46.456932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:46.474570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06da79-74f9-4494-91ca-98e5c85b04aa None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:46.487073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c59359-2504-4728-ae15-49a0e7c69ac2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:46.528111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:46.544155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:46.547174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.547674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.548625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.555116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:46.556304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.557669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.558155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.578973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:46.590580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:46.591503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:46.592314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 134/261] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:52:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:52:46.617071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:46.648987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:46.651110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:46.771552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 128/262] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:46.778611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:46.825556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:46.888537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c9f2f-b0ab-4138-8e66-00789a23886b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:46.921647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84719d3f-7150-46a5-a7be-986bb34afbdf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:46.929290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:46.929290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.929290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.929290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.929671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:46.929671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:46.929671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:46.937874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 135/263] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:52:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:46.943660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:46.952768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:46.986433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:46.987900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:46.988397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:46.988871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:46.994006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:46.995757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:46.997775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 129/264] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:46 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:47.004732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:47.053894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08008d88-e12c-430b-b53a-c3d10672cda2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:47.095262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:47.096054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:47.096856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:47.097282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] PATH_INFO: `/v3/users/35572f1c637c4421959b0ba9c478df96` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:47.099689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] RBAC: Authorizing `identity:get_user(user_id=35572f1c637c4421959b0ba9c478df96)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:47.101140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:47.103878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 136/265] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:46 2022] GET /identity/v3/users/35572f1c637c4421959b0ba9c478df96 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:47.109450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:47.174743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:47.188946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313be5e-4dfc-4639-9fd7-8d49152964fc None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:47.249650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:47.260688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:47.261383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:47.261668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:47.261946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:47.262785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=35572f1c637c4421959b0ba9c478df96) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:47.263100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:47.264535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:47.272361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 130/266] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:47 2022] GET /identity/v3/role_assignments?user.id=35572f1c637c4421959b0ba9c478df96&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:47.300576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa364c-c433-4eeb-831c-449856435c39 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:47.359582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:47.360161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:47.360448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:47.360727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:47.458287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:47.459774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:47.564029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:47.595684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 137/267] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:47.601440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:47.767537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d13ee-90bd-42c4-bc25-70b3cf0514e8 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:47.816921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d672d15-fce2-4ffe-bf86-f0a4492dd161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:47.818670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d672d15-fce2-4ffe-bf86-f0a4492dd161 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:47.819180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d672d15-fce2-4ffe-bf86-f0a4492dd161 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:47.819655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d672d15-fce2-4ffe-bf86-f0a4492dd161 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:47.821052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d672d15-fce2-4ffe-bf86-f0a4492dd161 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:47.822716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d672d15-fce2-4ffe-bf86-f0a4492dd161 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:47.825605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 131/268] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:47 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:47.830456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:47.842817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16183873-7d92-4478-9afc-ca44a719f113 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:47.843410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16183873-7d92-4478-9afc-ca44a719f113 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:47.843886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16183873-7d92-4478-9afc-ca44a719f113 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:47.845783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 132/269] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:47.851955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ba5a-2cb9-4eb0-b689-41a87957debb None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:47.852491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ba5a-2cb9-4eb0-b689-41a87957debb None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:47.852971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833ba5a-2cb9-4eb0-b689-41a87957debb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:47.877195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2833ba5a-2cb9-4eb0-b689-41a87957debb None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:47.879274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2833ba5a-2cb9-4eb0-b689-41a87957debb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:47.901749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:47.999394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 133/270] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:48.022099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50eb3e3-bf6b-4402-9df8-8c8cfdad3072 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:48.067468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833ba5a-2cb9-4eb0-b689-41a87957debb None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:48.086530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:48.087564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.088992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.089582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.092210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] RBAC: Adding query filter params (role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=35572f1c637c4421959b0ba9c478df96) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:48.094258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:48.096001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:48.108578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 138/271] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:47 2022] GET /identity/v3/role_assignments?user.id=35572f1c637c4421959b0ba9c478df96&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 608 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:48.198806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833ba5a-2cb9-4eb0-b689-41a87957debb None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:48.240900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fdbe4-a212-4aa3-ad24-dff215b8f651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:48.241759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fdbe4-a212-4aa3-ad24-dff215b8f651 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.242045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fdbe4-a212-4aa3-ad24-dff215b8f651 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.242520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fdbe4-a212-4aa3-ad24-dff215b8f651 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.259375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3fdbe4-a212-4aa3-ad24-dff215b8f651 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-2f3fdbe4-a212-4aa3-ad24-dff215b8f651 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:48.267595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:48.269157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:48.269999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:48.274220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 134/272] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:48.279923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:48.300395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776db1c-6624-48ad-9409-c3c094050325 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.300787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776db1c-6624-48ad-9409-c3c094050325 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.301070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4776db1c-6624-48ad-9409-c3c094050325 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.302657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 135/273] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:48.308040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e999f0f-8897-450d-9857-4e4b04d7a899 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.308326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e999f0f-8897-450d-9857-4e4b04d7a899 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.308604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e999f0f-8897-450d-9857-4e4b04d7a899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.338432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1e999f0f-8897-450d-9857-4e4b04d7a899 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:48.339646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1e999f0f-8897-450d-9857-4e4b04d7a899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:48.359052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:48.369194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:48.479362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 136/274] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:48.486372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d721f-bf1d-4d97-b184-b036bda30c92 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:48.515102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e999f0f-8897-450d-9857-4e4b04d7a899 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:48.528424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:48.529545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.530147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.530693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.531705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:48.533163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:48.535865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:48.547109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 139/275] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:48.553695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:48.647988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e999f0f-8897-450d-9857-4e4b04d7a899 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:48.687177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:48.688902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:48.689983 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba418a5-6c51-4d45-9912-46f9cce6d0d2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:48.690979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.691605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.692245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.695391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:48.697099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:48.699732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 137/276] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:48.711571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:48.735452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:48.742616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058c2d-d0dd-4418-a4ea-cf42e6cb8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:48.743937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058c2d-d0dd-4418-a4ea-cf42e6cb8986 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.750480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058c2d-d0dd-4418-a4ea-cf42e6cb8986 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.750480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058c2d-d0dd-4418-a4ea-cf42e6cb8986 None admin] PATH_INFO: `/v3/users/38671732ae2f4788b702215d932f36e7` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.760534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a058c2d-d0dd-4418-a4ea-cf42e6cb8986 None admin] RBAC: Authorizing `identity:get_user(user_id=38671732ae2f4788b702215d932f36e7)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:48.762383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a058c2d-d0dd-4418-a4ea-cf42e6cb8986 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:48.764389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 140/277] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:48 2022] GET /identity/v3/users/38671732ae2f4788b702215d932f36e7 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:48.767483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8c3df-e252-4008-8f9e-196b87583efe None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.767908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8c3df-e252-4008-8f9e-196b87583efe None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.768382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b8c3df-e252-4008-8f9e-196b87583efe None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.771019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 141/278] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:48.772208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8c3df-e252-4008-8f9e-196b87583efe None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:48.833589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb797c-4d5c-4a32-b731-fcb358120393 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:48.873806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:48.874473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.874755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.875026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.875736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:48.877178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:48.880158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b8c3df-e252-4008-8f9e-196b87583efe None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:48.914254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b0083d-6cca-42ac-85ee-0402716b4816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:48.914904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0083d-6cca-42ac-85ee-0402716b4816 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.915183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0083d-6cca-42ac-85ee-0402716b4816 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.915471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0083d-6cca-42ac-85ee-0402716b4816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.916280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0083d-6cca-42ac-85ee-0402716b4816 None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, user.id=38671732ae2f4788b702215d932f36e7, scope.system=all) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:48.916588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0083d-6cca-42ac-85ee-0402716b4816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:48.918075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b0083d-6cca-42ac-85ee-0402716b4816 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:48.922998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 142/279] 10.222.0.35 () {56 vars in 1308 bytes} [Tue May 24 09:52:48 2022] GET /identity/v3/role_assignments?user.id=38671732ae2f4788b702215d932f36e7&scope.system=all&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:48.924146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364d42-a7a7-42ef-b00a-1a720ff3f276 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:48.926045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364d42-a7a7-42ef-b00a-1a720ff3f276 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:48.926045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c364d42-a7a7-42ef-b00a-1a720ff3f276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:48.949170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6c364d42-a7a7-42ef-b00a-1a720ff3f276 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:48.950568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6c364d42-a7a7-42ef-b00a-1a720ff3f276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:48.973841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:48.986875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 138/280] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:52:48 2022] POST /identity/v3/users => generated 293 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:52:49.054221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 143/281] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:49.078365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.165360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52186f-ddb3-4e32-96ea-771d27ee6bf8 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.165360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52186f-ddb3-4e32-96ea-771d27ee6bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.172228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a52186f-ddb3-4e32-96ea-771d27ee6bf8 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.172228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 144/282] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:49.172415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:49.174112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859070fd-cfd6-4417-8b17-e50ae80e9801 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.174399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859070fd-cfd6-4417-8b17-e50ae80e9801 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.175199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859070fd-cfd6-4417-8b17-e50ae80e9801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.192980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-859070fd-cfd6-4417-8b17-e50ae80e9801 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:49.194224 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-859070fd-cfd6-4417-8b17-e50ae80e9801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:49.209474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb220a4f-f7e7-40c8-a849-81d9156e2ee2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:49.257414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:49.258214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.258504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.258784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.274027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:49.282306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 145/283] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:49.286165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:49.287073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:49.287896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 139/284] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:49.294145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859070fd-cfd6-4417-8b17-e50ae80e9801 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.306577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.397677 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:49.418809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859070fd-cfd6-4417-8b17-e50ae80e9801 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:49.432798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe4a22d-bfdf-471c-852f-c23b404e0bd3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:49.469674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:49.470467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.470755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.471030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.472397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:49.472701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:49.474144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:49.475370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:49.482028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:49.486939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.486939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.486939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.487212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 146/285] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:49.494451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.496012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:49.506989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:49.507912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:49.508748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:49.509410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 140/286] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:49.518862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.609793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3eb37c-00a6-4a92-abe0-ff68cf04c324 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:49.656001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:49.656891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.657174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.659034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] PATH_INFO: `/v3/users/673c8c67167348cebefb65699ea0f0dc` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.661937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] RBAC: Authorizing `identity:get_user(user_id=673c8c67167348cebefb65699ea0f0dc)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:49.663404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:49.665057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 147/287] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:49 2022] GET /identity/v3/users/673c8c67167348cebefb65699ea0f0dc => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:49.671765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.701570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:49.719800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5398b3-1f94-4ef9-8cec-6d21749345ab None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:49.779230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:49.780155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.780442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.780719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.781514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:49.781821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:49.784132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:49.794687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 141/288] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:49.800780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.848307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d56be4d-9aff-461a-93aa-96ca80ed5f2f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:49.885172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:49.886077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:49.886363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:49.887119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:49.887979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:49.889555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:49.891918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 148/289] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:49 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:49.897997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:49.901389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:49.982969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:50.000881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea92887-e7ba-485f-b261-d73f0ace5186 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:50.011693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfe243-0def-4127-a979-db56a3a4aee4 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:50.038131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:50.039201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:50.039586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:50.039965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119/users/673c8c67167348cebefb65699ea0f0dc/roles/1268b0be014043318348a2854238939a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:50.042126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] RBAC: Authorizing `identity:create_grant(project_id=db7757ba61ab423fa983d6fa71116119, user_id=673c8c67167348cebefb65699ea0f0dc, role_id=1268b0be014043318348a2854238939a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:50.043686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:50.055940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:50.057146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:50.057771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:50.058247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] PATH_INFO: `/v3/users/353141c307974b078a700bb724209420` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:50.059649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] RBAC: Authorizing `identity:get_user(user_id=353141c307974b078a700bb724209420)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:50.061305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:50.064342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 142/290] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:49 2022] GET /identity/v3/users/353141c307974b078a700bb724209420 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:50.069239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:50.119074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 149/291] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:52:49 2022] PUT /identity/v3/projects/db7757ba61ab423fa983d6fa71116119/users/673c8c67167348cebefb65699ea0f0dc/roles/1268b0be014043318348a2854238939a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:52:50.208212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690d4cc-3ba2-4dd1-809f-b24c76404020 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:50.231388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:50.254976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:50.255634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:50.255922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:50.256202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:50.349729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:50.351210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:50.470568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 143/292] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:52:50.476541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:50.590254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1873a8f4-3c76-4f2e-bd2d-e8f19566c3ab None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:50.632575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34eb09-41be-4702-aaee-2913c48882eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:50.633906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34eb09-41be-4702-aaee-2913c48882eb None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:50.634399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34eb09-41be-4702-aaee-2913c48882eb None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:50.634878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34eb09-41be-4702-aaee-2913c48882eb None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:50.638567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34eb09-41be-4702-aaee-2913c48882eb None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:50.640247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34eb09-41be-4702-aaee-2913c48882eb None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:50.642473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 150/293] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:50 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:50.649106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:50.690904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:50.800529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690a613-7f8d-4c6a-b9e0-b92a76bd61e6 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:50.848900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0beb259-21d7-4fc2-b01d-7699794160cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:50.849564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beb259-21d7-4fc2-b01d-7699794160cc None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:50.849847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beb259-21d7-4fc2-b01d-7699794160cc None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:50.850116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0beb259-21d7-4fc2-b01d-7699794160cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:50.851003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0beb259-21d7-4fc2-b01d-7699794160cc None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=353141c307974b078a700bb724209420) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:50.851314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0beb259-21d7-4fc2-b01d-7699794160cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:50.852732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0beb259-21d7-4fc2-b01d-7699794160cc None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:50.862202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 144/294] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:50 2022] GET /identity/v3/role_assignments?user.id=353141c307974b078a700bb724209420&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:51.416953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb32a-a65c-4372-80c1-02b906831a41 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:51.417338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb32a-a65c-4372-80c1-02b906831a41 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:51.417457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877cb32a-a65c-4372-80c1-02b906831a41 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:51.419070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 151/295] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:51.424448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174ed8-742d-438b-8619-122f47f569ae None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:51.425553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174ed8-742d-438b-8619-122f47f569ae None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:51.425730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b174ed8-742d-438b-8619-122f47f569ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:51.445198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8b174ed8-742d-438b-8619-122f47f569ae None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:51.446416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8b174ed8-742d-438b-8619-122f47f569ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:51.520693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 145/296] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:51.544672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb32a-a65c-4372-80c1-02b906831a41 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:51.557539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:51.662206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877cb32a-a65c-4372-80c1-02b906831a41 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:51.700337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df849e-52cc-4e7d-a353-729de4f34749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:51.701133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df849e-52cc-4e7d-a353-729de4f34749 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:51.701442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df849e-52cc-4e7d-a353-729de4f34749 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:51.701717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df849e-52cc-4e7d-a353-729de4f34749 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:51.712804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df849e-52cc-4e7d-a353-729de4f34749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-a6df849e-52cc-4e7d-a353-729de4f34749 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:51.720619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:51.722250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:51.723944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 152/297] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:51.728609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174ed8-742d-438b-8619-122f47f569ae None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:51.748290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:51.837288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b174ed8-742d-438b-8619-122f47f569ae None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:51.846531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ff1fb-147a-4137-ba7e-6941c0fd4b7f None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:51.846920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ff1fb-147a-4137-ba7e-6941c0fd4b7f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:51.847274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ff1fb-147a-4137-ba7e-6941c0fd4b7f None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:51.848897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 153/298] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:51.853842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45c402-f8fe-4f42-a9df-222874540d93 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:51.854129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45c402-f8fe-4f42-a9df-222874540d93 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:51.854487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45c402-f8fe-4f42-a9df-222874540d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:51.872991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9a45c402-f8fe-4f42-a9df-222874540d93 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:51.874570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9a45c402-f8fe-4f42-a9df-222874540d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:51.875718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:51.876668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:51.877139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:51.877658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:51.879306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] RBAC: Adding query filter params (name=member) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:51.879886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:51.881589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:51.891903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 146/299] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:51.897915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:51.958357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 154/300] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:51.958878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:51.981352 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45c402-f8fe-4f42-a9df-222874540d93 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:51.996161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:52.029134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92964af-d043-4323-8819-b90cb8bb6e29 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:52.076393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255d157-d91a-4412-8f28-abb600065042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:52.077848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255d157-d91a-4412-8f28-abb600065042 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.078132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255d157-d91a-4412-8f28-abb600065042 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.078680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255d157-d91a-4412-8f28-abb600065042 None admin] PATH_INFO: `/v3/users/38671732ae2f4788b702215d932f36e7` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.079907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255d157-d91a-4412-8f28-abb600065042 None admin] RBAC: Authorizing `identity:get_user(user_id=38671732ae2f4788b702215d932f36e7)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:52.082405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255d157-d91a-4412-8f28-abb600065042 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:52.084175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 147/301] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:51 2022] GET /identity/v3/users/38671732ae2f4788b702215d932f36e7 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:52.085222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d7db1-f8f5-41bb-a837-fba3930cbb3f None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.085579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d7db1-f8f5-41bb-a837-fba3930cbb3f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.085795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d7db1-f8f5-41bb-a837-fba3930cbb3f None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.087133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 148/302] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:52.091544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d7db1-f8f5-41bb-a837-fba3930cbb3f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:52.098275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45c402-f8fe-4f42-a9df-222874540d93 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:52.134139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da8183-5f16-4794-91a3-4f609f0f3f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:52.134937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da8183-5f16-4794-91a3-4f609f0f3f92 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.135217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da8183-5f16-4794-91a3-4f609f0f3f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.135849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da8183-5f16-4794-91a3-4f609f0f3f92 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.136678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da8183-5f16-4794-91a3-4f609f0f3f92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:52.138156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da8183-5f16-4794-91a3-4f609f0f3f92 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:52.144902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 155/303] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:52.146148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0278f69-aaac-43f8-9503-3a7eb750eb6c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.146437 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0278f69-aaac-43f8-9503-3a7eb750eb6c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.146719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0278f69-aaac-43f8-9503-3a7eb750eb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.164100 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f0278f69-aaac-43f8-9503-3a7eb750eb6c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:52.166185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f0278f69-aaac-43f8-9503-3a7eb750eb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:52.207134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d7db1-f8f5-41bb-a837-fba3930cbb3f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:52.248263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:52.254981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:52.255846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.256122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.256391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] PATH_INFO: `/v3/system/users/38671732ae2f4788b702215d932f36e7/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.257385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 156/304] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:52.258089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0278f69-aaac-43f8-9503-3a7eb750eb6c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:52.265879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=38671732ae2f4788b702215d932f36e7, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:52.267582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:52.353317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 149/305] 10.222.0.35 () {58 vars in 1334 bytes} [Tue May 24 09:52:52 2022] PUT /identity/v3/system/users/38671732ae2f4788b702215d932f36e7/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:52.361468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:52.363674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0278f69-aaac-43f8-9503-3a7eb750eb6c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:52.389685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:52.415791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:52.416486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.416769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.417215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] PATH_INFO: `/v3/groups` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.418407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:52.420374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:52.479924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e9358-803c-4b3a-9ee6-e4925e84b548 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:52.484097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 157/306] 10.222.0.35 () {60 vars in 1134 bytes} [Tue May 24 09:52:52 2022] POST /identity/v3/groups => generated 229 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:52:52.534707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:52.535596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.535873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.536148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.550413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:52.558738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:52.559685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:52.560489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 150/307] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:52.566854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:52.639788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:52.691923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5765ddb-d831-46b2-89bb-c325a7babfce None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:52.728871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4723e0f8-6213-4f9e-9a73-276abcb2eb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:52.729593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723e0f8-6213-4f9e-9a73-276abcb2eb75 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.729879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723e0f8-6213-4f9e-9a73-276abcb2eb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.730161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4723e0f8-6213-4f9e-9a73-276abcb2eb75 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.730929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723e0f8-6213-4f9e-9a73-276abcb2eb75 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:52.731238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723e0f8-6213-4f9e-9a73-276abcb2eb75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:52.732630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4723e0f8-6213-4f9e-9a73-276abcb2eb75 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:52.742911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 158/308] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:52.748214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:52.819669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:52.824825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8fb64-09dc-406a-a315-f40805cd036f None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.824976 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8fb64-09dc-406a-a315-f40805cd036f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.825233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8fb64-09dc-406a-a315-f40805cd036f None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.828008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 159/309] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:52.832740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b690-ad72-42f8-80d3-178674c7c73e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.833024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b690-ad72-42f8-80d3-178674c7c73e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.833338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254b690-ad72-42f8-80d3-178674c7c73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.851804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b254b690-ad72-42f8-80d3-178674c7c73e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:52.853305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b254b690-ad72-42f8-80d3-178674c7c73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:52.858436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efac6b8-1a21-4141-9aa0-be9933619cee None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:52.894771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:52.896047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:52.896471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:52.896884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] PATH_INFO: `/v3/users/96c68f960bfc40d7a118c93f860c0fa5` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:52.913534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] RBAC: Authorizing `identity:get_user(user_id=96c68f960bfc40d7a118c93f860c0fa5)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:52.915228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:52.917400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 151/310] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:52 2022] GET /identity/v3/users/96c68f960bfc40d7a118c93f860c0fa5 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:52.922616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:52.945918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 160/311] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:52.968751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b690-ad72-42f8-80d3-178674c7c73e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:53.033516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:53.044833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a14e988-39cf-4be6-973c-5ff4f3b0d6c0 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:53.088966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:53.094420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:53.095100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:53.095385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:53.095662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:53.097075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] RBAC: Adding query filter params (role.id=1268b0be014043318348a2854238939a, user.id=96c68f960bfc40d7a118c93f860c0fa5, scope.system=all) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:53.097707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:53.099263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:53.099541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254b690-ad72-42f8-80d3-178674c7c73e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:53.113850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 152/312] 10.222.0.35 () {56 vars in 1308 bytes} [Tue May 24 09:52:52 2022] GET /identity/v3/role_assignments?user.id=96c68f960bfc40d7a118c93f860c0fa5&scope.system=all&role.id=1268b0be014043318348a2854238939a => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:53.141328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:53.142150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:53.142431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:53.142707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:53.155704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:53.164464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:53.175512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:53.176837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 161/313] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:52:53.177583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:53.296288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7de71-1a47-4ba9-aed5-89167f4d1a84 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:53.334758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:53.335498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:53.335784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:53.336065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:53.336875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:53.337188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:53.338615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:53.348086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 153/314] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:53.353564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:53.471050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553dc5e-792d-4c69-b410-463b35d83071 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:53.481587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:53.507720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:53.508605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:53.508886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:53.509158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] PATH_INFO: `/v3/users/673c8c67167348cebefb65699ea0f0dc` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:53.510396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] RBAC: Authorizing `identity:get_user(user_id=673c8c67167348cebefb65699ea0f0dc)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:53.511916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:53.513888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 162/315] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:53 2022] GET /identity/v3/users/673c8c67167348cebefb65699ea0f0dc => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:53.520142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:53.597667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:53.634853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa109062-0dfa-46e8-82cf-c1247cf43b7f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:53.672348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:53.673402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:53.673894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:53.674395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:53.750603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:53.752412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:53.809934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:53.863294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 154/316] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:53.869469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:53.978024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94323bc6-f0a1-442d-9cbb-812d76e9125d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:54.012117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56907ca8-e7fa-408f-89ef-4fc5a04df5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:54.013042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56907ca8-e7fa-408f-89ef-4fc5a04df5de None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:54.013354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56907ca8-e7fa-408f-89ef-4fc5a04df5de None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:54.013643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56907ca8-e7fa-408f-89ef-4fc5a04df5de None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:54.014817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56907ca8-e7fa-408f-89ef-4fc5a04df5de None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:54.016533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56907ca8-e7fa-408f-89ef-4fc5a04df5de None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:54.018295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 163/317] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:53 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:54.024054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:54.133006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf230a-52eb-4e2c-b303-65f0df4b871c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:54.170437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e3029-698d-402f-8bdb-d17dd4798c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:54.171116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3029-698d-402f-8bdb-d17dd4798c6a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:54.171404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3029-698d-402f-8bdb-d17dd4798c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:54.173242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e3029-698d-402f-8bdb-d17dd4798c6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:54.173242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e3029-698d-402f-8bdb-d17dd4798c6a None admin] RBAC: Adding query filter params (role.id=1268b0be014043318348a2854238939a, scope.project.id=db7757ba61ab423fa983d6fa71116119, user.id=673c8c67167348cebefb65699ea0f0dc) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:54.173242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e3029-698d-402f-8bdb-d17dd4798c6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:54.174392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e3029-698d-402f-8bdb-d17dd4798c6a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:54.182248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 155/318] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:52:54 2022] GET /identity/v3/role_assignments?user.id=673c8c67167348cebefb65699ea0f0dc&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=1268b0be014043318348a2854238939a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:54.683243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f445fb-7013-4818-9af3-726216e012c2 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:54.683645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f445fb-7013-4818-9af3-726216e012c2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:54.683912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f445fb-7013-4818-9af3-726216e012c2 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:54.685343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 164/319] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:54.690874 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b9d0a-5253-40bc-9c72-1670b6b66f06 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:54.691504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b9d0a-5253-40bc-9c72-1670b6b66f06 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:54.691504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1b9d0a-5253-40bc-9c72-1670b6b66f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:54.708083 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ae1b9d0a-5253-40bc-9c72-1670b6b66f06 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:54.709290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ae1b9d0a-5253-40bc-9c72-1670b6b66f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:54.720153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:54.779387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 156/320] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:54.803072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f445fb-7013-4818-9af3-726216e012c2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:54.818516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13fdaf-a432-447b-8110-c300695f7007 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:54.818816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13fdaf-a432-447b-8110-c300695f7007 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:54.819091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13fdaf-a432-447b-8110-c300695f7007 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:54.820488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 157/321] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:52:54.826300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2beb7-ed35-430c-96ab-9416e0cfec7c None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:54.826589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2beb7-ed35-430c-96ab-9416e0cfec7c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:54.826873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2beb7-ed35-430c-96ab-9416e0cfec7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:54.844888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-68f2beb7-ed35-430c-96ab-9416e0cfec7c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:54.846106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-68f2beb7-ed35-430c-96ab-9416e0cfec7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:54.858314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:54.916368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 158/322] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:54.918249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f445fb-7013-4818-9af3-726216e012c2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:54.939582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2beb7-ed35-430c-96ab-9416e0cfec7c None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:54.958601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:54.959914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:54.960218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:54.960498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:54.971367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:54.979308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:54.980235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:54.986269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 165/323] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:54.988797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:55.027577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:55.053491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2beb7-ed35-430c-96ab-9416e0cfec7c None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:55.093107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:55.093935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.094221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.094496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.095657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:55.098155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:55.099973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 159/324] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:52:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:55.105104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b360d664-3e36-47e8-9846-db7804a5e6ec None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:55.105551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:55.107485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:55.139897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:55.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:55.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:55.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:55.156629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 166/325] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:55.160878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:55.228364 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fc08a-e06d-495e-93b7-ba212e4970fe None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:55.266769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aae288-7476-44c2-8867-5c88e77294de None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:55.267779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:55.269234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.269560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.269803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] PATH_INFO: `/v3/groups` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.270967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:55.272408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:55.303575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:55.304443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.304721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.304988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] PATH_INFO: `/v3/users/38671732ae2f4788b702215d932f36e7` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.306189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] RBAC: Authorizing `identity:get_user(user_id=38671732ae2f4788b702215d932f36e7)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:55.307619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:55.309231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 167/326] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:55 2022] GET /identity/v3/users/38671732ae2f4788b702215d932f36e7 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:55.314864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:55.412539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd274ccc-8677-47e2-86e9-ed458b837fbf None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:55.445648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909c303-8d6c-4b9e-82f9-8758ac1c3af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:55.448491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909c303-8d6c-4b9e-82f9-8758ac1c3af2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.448491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909c303-8d6c-4b9e-82f9-8758ac1c3af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.448491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909c303-8d6c-4b9e-82f9-8758ac1c3af2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.448491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909c303-8d6c-4b9e-82f9-8758ac1c3af2 None admin] RBAC: Adding query filter params (role.id=af0d03236dab45f3a150b6281f50bb80, user.id=38671732ae2f4788b702215d932f36e7, scope.system=all) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:55.448491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909c303-8d6c-4b9e-82f9-8758ac1c3af2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:55.449436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909c303-8d6c-4b9e-82f9-8758ac1c3af2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:55.461605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 168/327] 10.222.0.35 () {56 vars in 1308 bytes} [Tue May 24 09:52:55 2022] GET /identity/v3/role_assignments?user.id=38671732ae2f4788b702215d932f36e7&scope.system=all&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 510 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:55.462819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6676b-3611-4e60-9c71-c2a894561b6b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.463106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6676b-3611-4e60-9c71-c2a894561b6b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.463386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6676b-3611-4e60-9c71-c2a894561b6b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.464622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 169/328] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:55.470685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0274-1122-4dc2-a31e-cd182d7ab5fa None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.470976 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0274-1122-4dc2-a31e-cd182d7ab5fa None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.471569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae0274-1122-4dc2-a31e-cd182d7ab5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.490260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-99ae0274-1122-4dc2-a31e-cd182d7ab5fa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:55.491395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-99ae0274-1122-4dc2-a31e-cd182d7ab5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:55.519913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 160/329] 10.222.0.35 () {60 vars in 1134 bytes} [Tue May 24 09:52:55 2022] POST /identity/v3/groups => generated 226 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:52:55.548342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:55.573547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 170/330] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:55.597480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:55.601606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:55.718714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5e64-7733-4b03-aeaf-ddee22f38475 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:55.754844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:55.755573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.755858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.756144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.766639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:55.774920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:55.776034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:55.776857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 161/331] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:55.781899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae0274-1122-4dc2-a31e-cd182d7ab5fa None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:55.807446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:55.884379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae0274-1122-4dc2-a31e-cd182d7ab5fa None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:55.921969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:55.922722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:55.923315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:55.923315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:55.923943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:55.924252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:55.926242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:55.937619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 171/332] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:55.942060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:56.011905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:56.054766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a17-2a71-46ef-9d0a-8a85be24cc78 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:56.092388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585ce14f-f292-4e96-9e5b-c6823e9571a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:56.093331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ce14f-f292-4e96-9e5b-c6823e9571a1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:56.093580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ce14f-f292-4e96-9e5b-c6823e9571a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:56.093858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585ce14f-f292-4e96-9e5b-c6823e9571a1 None admin] PATH_INFO: `/v3/users/96c68f960bfc40d7a118c93f860c0fa5` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:56.095166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ce14f-f292-4e96-9e5b-c6823e9571a1 None admin] RBAC: Authorizing `identity:get_user(user_id=96c68f960bfc40d7a118c93f860c0fa5)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:56.097392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585ce14f-f292-4e96-9e5b-c6823e9571a1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:56.099401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 162/333] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:52:55 2022] GET /identity/v3/users/96c68f960bfc40d7a118c93f860c0fa5 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:56.104934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:56.213014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:56.230376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885713b-5d84-4d07-afd6-e263aa2f383f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:56.267316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b70df91-9b19-4e2d-8c5b-6b358a9e2420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:56.268169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70df91-9b19-4e2d-8c5b-6b358a9e2420 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:56.269027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70df91-9b19-4e2d-8c5b-6b358a9e2420 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:56.269806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b70df91-9b19-4e2d-8c5b-6b358a9e2420 None admin] PATH_INFO: `/v3/system/users/96c68f960bfc40d7a118c93f860c0fa5/roles/1268b0be014043318348a2854238939a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:56.271748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70df91-9b19-4e2d-8c5b-6b358a9e2420 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=96c68f960bfc40d7a118c93f860c0fa5, role_id=1268b0be014043318348a2854238939a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:56.273147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b70df91-9b19-4e2d-8c5b-6b358a9e2420 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:56.336342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 172/334] 10.222.0.35 () {58 vars in 1334 bytes} [Tue May 24 09:52:56 2022] PUT /identity/v3/system/users/96c68f960bfc40d7a118c93f860c0fa5/roles/1268b0be014043318348a2854238939a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:52:57.984443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284e432-d03e-4bbb-9e0a-dfd8709df62e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:57.984979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284e432-d03e-4bbb-9e0a-dfd8709df62e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:57.984979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9284e432-d03e-4bbb-9e0a-dfd8709df62e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:57.986658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 163/335] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:57.992566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf56448-3f38-4356-9586-a885cbd5ea54 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:57.993006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf56448-3f38-4356-9586-a885cbd5ea54 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:57.993200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf56448-3f38-4356-9586-a885cbd5ea54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:58.011455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-aaf56448-3f38-4356-9586-a885cbd5ea54 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:58.012659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-aaf56448-3f38-4356-9586-a885cbd5ea54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:58.100011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 173/336] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:58.138493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284e432-d03e-4bbb-9e0a-dfd8709df62e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:58.279989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9284e432-d03e-4bbb-9e0a-dfd8709df62e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:58.319576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:58.325318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:58.326142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:58.326430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:58.326710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:58.344144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:58.352982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:58.353933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:52:58.354755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 164/337] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:58.360212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf56448-3f38-4356-9586-a885cbd5ea54 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:58.469494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:58.511685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf56448-3f38-4356-9586-a885cbd5ea54 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:58.562340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a18c7-3c97-4e3a-8e98-79ac8b98fc94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:58.562944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a18c7-3c97-4e3a-8e98-79ac8b98fc94 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:58.563221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a18c7-3c97-4e3a-8e98-79ac8b98fc94 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:58.563492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a18c7-3c97-4e3a-8e98-79ac8b98fc94 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:58.564257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a18c7-3c97-4e3a-8e98-79ac8b98fc94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:58.564560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a18c7-3c97-4e3a-8e98-79ac8b98fc94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:58.566704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296a18c7-3c97-4e3a-8e98-79ac8b98fc94 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:58.582498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 174/338] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:58.588696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:58.734213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6d63d-0c57-45e9-9e9b-3e52e273ec04 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:58.780831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:58.781526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:58.781815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:58.782095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:58.844696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fc7af-dc6a-4a4c-b9f1-49d2fa76bf21 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:58.845353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fc7af-dc6a-4a4c-b9f1-49d2fa76bf21 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:58.845870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fc7af-dc6a-4a4c-b9f1-49d2fa76bf21 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:58.850998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:58.851969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 175/339] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:58.856461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69abf35-c775-40d9-ad2c-e151649e2ba6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:58.857003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69abf35-c775-40d9-ad2c-e151649e2ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:58.857551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69abf35-c775-40d9-ad2c-e151649e2ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:58.860075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:58.861485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:58.895217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f69abf35-c775-40d9-ad2c-e151649e2ba6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:58.897678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f69abf35-c775-40d9-ad2c-e151649e2ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:58.989970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 165/340] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:52:58.991782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 176/341] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:52:59.006625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:59.022360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69abf35-c775-40d9-ad2c-e151649e2ba6 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:59.119301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:59.139312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:59.149726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4206f52-e101-4957-8ddf-173454173bae None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:59.162618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69abf35-c775-40d9-ad2c-e151649e2ba6 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:59.202622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8f7a2-2ed3-41c1-86d8-6b946bb83f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:59.203619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8f7a2-2ed3-41c1-86d8-6b946bb83f6e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.203970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8f7a2-2ed3-41c1-86d8-6b946bb83f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.204318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8f7a2-2ed3-41c1-86d8-6b946bb83f6e None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.205613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8f7a2-2ed3-41c1-86d8-6b946bb83f6e None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:59.209288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8f7a2-2ed3-41c1-86d8-6b946bb83f6e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:59.212940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 166/342] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:52:59 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:59.214778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323afca3-e282-4338-83f8-d73422645c83 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.215132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323afca3-e282-4338-83f8-d73422645c83 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.215482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323afca3-e282-4338-83f8-d73422645c83 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.217557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 167/343] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:59.220301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323afca3-e282-4338-83f8-d73422645c83 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:59.249999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416cbc9-52a5-4e8f-891b-c2a1e092667b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:59.251090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cbc9-52a5-4e8f-891b-c2a1e092667b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.251522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cbc9-52a5-4e8f-891b-c2a1e092667b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.252021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cbc9-52a5-4e8f-891b-c2a1e092667b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.279089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e416cbc9-52a5-4e8f-891b-c2a1e092667b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-e416cbc9-52a5-4e8f-891b-c2a1e092667b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:52:59.290514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:52:59.291424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:52:59.292225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 177/344] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:52:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:52:59.300864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a52b-7eca-48cf-987b-57d118c86dd7 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.301154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a52b-7eca-48cf-987b-57d118c86dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.301466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a52b-7eca-48cf-987b-57d118c86dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.337633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c301a52b-7eca-48cf-987b-57d118c86dd7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:52:59.338812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c301a52b-7eca-48cf-987b-57d118c86dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:52:59.408162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323afca3-e282-4338-83f8-d73422645c83 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:59.436263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 178/345] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:52:59.437136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301a52b-7eca-48cf-987b-57d118c86dd7 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:59.478225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:59.479223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.479574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.479851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.490914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:59.492385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:59.496106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 168/346] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:52:59 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:59.507721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:59.537647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:59.561500 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:59.609327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301a52b-7eca-48cf-987b-57d118c86dd7 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:59.676054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:59.676756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.677040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.677504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.678729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:59.679209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:59.681321 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:59.693416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15166ea7-08b0-4d23-99d4-ed71316e6048 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:59.697413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 179/347] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:52:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:52:59.699420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:59.743016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350177ad-f86c-4cad-8b92-5c5ff14fe028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:59.744155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350177ad-f86c-4cad-8b92-5c5ff14fe028 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.744898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350177ad-f86c-4cad-8b92-5c5ff14fe028 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.745624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350177ad-f86c-4cad-8b92-5c5ff14fe028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.746667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350177ad-f86c-4cad-8b92-5c5ff14fe028 None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:52:59.747159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350177ad-f86c-4cad-8b92-5c5ff14fe028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:52:59.749768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350177ad-f86c-4cad-8b92-5c5ff14fe028 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:52:59.758195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 169/348] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:52:59 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:52:59.760397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef16ee-d272-49a8-a6d7-366f26f7c790 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.760868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef16ee-d272-49a8-a6d7-366f26f7c790 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.765474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef16ee-d272-49a8-a6d7-366f26f7c790 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.768619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 170/349] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:52:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:52:59.772021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef16ee-d272-49a8-a6d7-366f26f7c790 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:52:59.897278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02511d7-b8bb-4a88-b576-83db713c2b41 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:52:59.954308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:59.971191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:52:59.985734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98071c70-321d-4a1f-927e-6b754113e572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:52:59.987814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98071c70-321d-4a1f-927e-6b754113e572 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:52:59.988223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98071c70-321d-4a1f-927e-6b754113e572 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:52:59.988508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98071c70-321d-4a1f-927e-6b754113e572 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:52:59.999608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef16ee-d272-49a8-a6d7-366f26f7c790 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:00.016261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98071c70-321d-4a1f-927e-6b754113e572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-98071c70-321d-4a1f-927e-6b754113e572 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:00.025684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:00.027077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:00.027917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:00.028637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 180/350] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:52:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:00.034102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c01280-0479-4ed3-8817-f39276a24bd4 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.036143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c01280-0479-4ed3-8817-f39276a24bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.036424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c01280-0479-4ed3-8817-f39276a24bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.057257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0d308-ae8b-4745-8090-77adc356d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:00.058495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0d308-ae8b-4745-8090-77adc356d57c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.059050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0d308-ae8b-4745-8090-77adc356d57c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.059542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0d308-ae8b-4745-8090-77adc356d57c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.063272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-00c01280-0479-4ed3-8817-f39276a24bd4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:00.064215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-00c01280-0479-4ed3-8817-f39276a24bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:00.147781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0d308-ae8b-4745-8090-77adc356d57c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:00.149639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0d308-ae8b-4745-8090-77adc356d57c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:00.193693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 181/351] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:00.194546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c01280-0479-4ed3-8817-f39276a24bd4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:00.278617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:00.293456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 171/352] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:00.295248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c659e29-f0fa-4c7d-9d8e-d67153058436 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.295705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c659e29-f0fa-4c7d-9d8e-d67153058436 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.296266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c659e29-f0fa-4c7d-9d8e-d67153058436 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.297961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 172/353] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:00.299024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c659e29-f0fa-4c7d-9d8e-d67153058436 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:00.325308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c01280-0479-4ed3-8817-f39276a24bd4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:00.340302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:00.361856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:00.363476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.363998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.364487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.365456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:00.365959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:00.367543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:00.376202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 182/354] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:00.377286 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:00.418605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c659e29-f0fa-4c7d-9d8e-d67153058436 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:00.458278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682eb58-dfcf-4108-a088-049cdbf6370e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:00.459464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eb58-dfcf-4108-a088-049cdbf6370e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.459940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eb58-dfcf-4108-a088-049cdbf6370e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.460405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682eb58-dfcf-4108-a088-049cdbf6370e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.478676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682eb58-dfcf-4108-a088-049cdbf6370e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-d682eb58-dfcf-4108-a088-049cdbf6370e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:00.493241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:00.494213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:00.495026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:00.497584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e7f18-9c8a-4f66-9ddb-6474002d5d84 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:00.498457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 173/355] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:00.499948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3be4e-c589-47df-a01c-c975004a173c None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.500477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3be4e-c589-47df-a01c-c975004a173c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.505300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3be4e-c589-47df-a01c-c975004a173c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.526485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-baf3be4e-c589-47df-a01c-c975004a173c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:00.528005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-baf3be4e-c589-47df-a01c-c975004a173c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:00.535279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:00.536080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.536364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.536643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.537879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:00.539327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:00.543439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 183/356] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:00.544422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:00.584045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:00.595212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:00.609649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 174/357] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:00.610993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3be4e-c589-47df-a01c-c975004a173c None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:00.671762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ab14c-5d84-4c6f-81a9-95a15d095912 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:00.714376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:00.715394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.715690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.716119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.742519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3be4e-c589-47df-a01c-c975004a173c None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:00.790120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a33360a-6c93-4e61-a32f-eee76fcc9589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:00.791020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33360a-6c93-4e61-a32f-eee76fcc9589 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.791462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33360a-6c93-4e61-a32f-eee76fcc9589 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.791742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33360a-6c93-4e61-a32f-eee76fcc9589 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.792658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33360a-6c93-4e61-a32f-eee76fcc9589 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:00.792974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33360a-6c93-4e61-a32f-eee76fcc9589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:00.794615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a33360a-6c93-4e61-a32f-eee76fcc9589 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:00.807680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 175/358] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:00.807680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea8948-61ac-4239-aeb4-a2032e548a88 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.807680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea8948-61ac-4239-aeb4-a2032e548a88 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.808007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea8948-61ac-4239-aeb4-a2032e548a88 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.809002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 176/359] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:00.810053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea8948-61ac-4239-aeb4-a2032e548a88 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:00.819828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:00.823786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:00.861525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:00.889678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:00.940981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea8948-61ac-4239-aeb4-a2032e548a88 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:00.959598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 184/360] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:00.960749 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:00.988814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:00.990200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:00.990496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:00.990835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:00.994315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:00.994315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:00.997434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 177/361] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:00.997434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.098126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae3b53-7dbb-4ec5-9693-b1d5ca83e008 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.124514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9d4bf-7198-4e8c-acc9-d4bce4d843c9 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.136286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:01.151519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47007efe-d7d8-48bc-b1f2-98825d16f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.155548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47007efe-d7d8-48bc-b1f2-98825d16f261 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.155548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47007efe-d7d8-48bc-b1f2-98825d16f261 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.155548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47007efe-d7d8-48bc-b1f2-98825d16f261 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.169810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47007efe-d7d8-48bc-b1f2-98825d16f261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:01.171937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.172597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.172881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.173155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-47007efe-d7d8-48bc-b1f2-98825d16f261 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:01.178496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:01.179424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 24 09:53:01.181601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:01.184552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 185/362] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:01.186174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c76d1-9542-4825-8d93-d6bd2c24e95f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.186468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c76d1-9542-4825-8d93-d6bd2c24e95f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.186751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4c76d1-9542-4825-8d93-d6bd2c24e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.192406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:01.207637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fc4c76d1-9542-4825-8d93-d6bd2c24e95f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:01.208881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fc4c76d1-9542-4825-8d93-d6bd2c24e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:01.271423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:01.272888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:01.299555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 186/363] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:01.305473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c76d1-9542-4825-8d93-d6bd2c24e95f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.409519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 178/364] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:01.411526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.424420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:01.437355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4c76d1-9542-4825-8d93-d6bd2c24e95f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.485779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:01.487176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.488371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.488655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.488929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.490325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:01.491795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:01.494178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 187/365] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:01.501106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.541996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174ab4-5a43-414c-a88c-8b620879ca4b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.586052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.586052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.586342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.586506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.587595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=db7757ba61ab423fa983d6fa71116119) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:01.591146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:01.591146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:01.597692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 179/366] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:01.599130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.634381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9599b58-c341-4db9-8451-1031b784c9b1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.682229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.682817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.683099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.683374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.684139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:01.684445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:01.685965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:01.695679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 188/367] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:01.696541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.705657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:01.736229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03406c2b-eb51-4aa7-a741-5fa85eba0f1d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.769174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:01.779245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.780131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.780421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.780698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.793205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:01.801817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:01.802757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:01.803546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 180/368] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:01.804702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.821681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db914683-9c5c-45e0-ba81-490838e34100 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.860227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.861095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.861420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.861747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.863849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:01.865326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:01.867225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 189/369] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:01.867815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.926863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717318a-e1ea-4249-af9e-1144b0932d46 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:01.966262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:01.973862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:01.975122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:01.975402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:01.975683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:01.976887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:01.979048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:01.981889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 181/370] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:01.983030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:01.996995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8237f2-6e7e-4f38-81ad-058290641759 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:02.033621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:02.038445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:02.039424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.039709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.039984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] PATH_INFO: `/v3/users/96c68f960bfc40d7a118c93f860c0fa5` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.042379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] RBAC: Authorizing `identity:get_user(user_id=96c68f960bfc40d7a118c93f860c0fa5)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:02.043953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:02.045994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 190/371] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/users/96c68f960bfc40d7a118c93f860c0fa5 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:02.046684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:02.132660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf3cd1-684d-4add-b167-c9d27bbea96d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:02.179073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63a7fe-11ce-4663-8756-7ccdf6e85ee5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:02.184396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:02.185098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.185429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.185743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.186531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:02.186849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:02.189036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:02.201558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 182/372] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:02.202504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:02.234346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98537c78-391f-4546-a422-3feefd74912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:02.235283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98537c78-391f-4546-a422-3feefd74912c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.235566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98537c78-391f-4546-a422-3feefd74912c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.235844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98537c78-391f-4546-a422-3feefd74912c None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.237091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98537c78-391f-4546-a422-3feefd74912c None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:02.239485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98537c78-391f-4546-a422-3feefd74912c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:02.241666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 191/373] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:02 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:02.242464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98537c78-391f-4546-a422-3feefd74912c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:02.305623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:02.329878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:02.351767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fde3ee-c79a-49e5-bf8d-b5ddfe7495ac None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:02.375054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98537c78-391f-4546-a422-3feefd74912c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:02.402034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:02.402709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.403136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.403372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.404213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:02.404530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:02.406069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:02.413368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 183/374] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:02 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:02.414205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:02.425128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:02.426281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.426675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.426961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.427901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] RBAC: Adding query filter params (role.id=1268b0be014043318348a2854238939a, user.id=96c68f960bfc40d7a118c93f860c0fa5, scope.system=all) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:02.428222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:02.429827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:02.448082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 192/375] 10.222.0.35 () {56 vars in 1308 bytes} [Tue May 24 09:53:02 2022] GET /identity/v3/role_assignments?user.id=96c68f960bfc40d7a118c93f860c0fa5&scope.system=all&role.id=1268b0be014043318348a2854238939a => generated 510 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:02.448912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:02.570255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898af4c-b1c4-4557-b38d-c2e1d6ba9f6f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:02.598877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202c330-d51b-43ab-afa4-1958893a2d65 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:02.609695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:02.634208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:02.634895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.635183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.635463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.642022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:02.665808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc172f86-5ebb-4a58-8387-142bbec588d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:02.666506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172f86-5ebb-4a58-8387-142bbec588d2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.666795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172f86-5ebb-4a58-8387-142bbec588d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.667078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc172f86-5ebb-4a58-8387-142bbec588d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.667949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172f86-5ebb-4a58-8387-142bbec588d2 None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=db7757ba61ab423fa983d6fa71116119) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:02.668268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172f86-5ebb-4a58-8387-142bbec588d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:02.673496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc172f86-5ebb-4a58-8387-142bbec588d2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:02.688037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 193/376] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:02 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:02.689811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cedb5a-39ce-494f-9aac-ae5ed0b480f5 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.690107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cedb5a-39ce-494f-9aac-ae5ed0b480f5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.690424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cedb5a-39ce-494f-9aac-ae5ed0b480f5 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.692501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 194/377] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:02.701736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c2479-a908-48a5-bb29-6913b297cbd1 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:02.701736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c2479-a908-48a5-bb29-6913b297cbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:02.701987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7c2479-a908-48a5-bb29-6913b297cbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:02.740211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5b7c2479-a908-48a5-bb29-6913b297cbd1 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:02.741506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5b7c2479-a908-48a5-bb29-6913b297cbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:02.802477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:02.803974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:02.877376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 195/378] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:02.922428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c2479-a908-48a5-bb29-6913b297cbd1 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:02.965454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 184/379] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:02.966991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:03.001672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:03.079514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e3186-a56a-4d73-a548-454cb36ab423 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:03.107606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7c2479-a908-48a5-bb29-6913b297cbd1 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:03.120476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:03.121384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:03.121671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:03.121948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] PATH_INFO: `/v3/projects/d39d97f2faca4faba12c96da090b5017` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:03.123157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] RBAC: Authorizing `identity:get_project(project_id=d39d97f2faca4faba12c96da090b5017)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:03.124603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:03.126647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 185/380] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:02 2022] GET /identity/v3/projects/d39d97f2faca4faba12c96da090b5017 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:03.132377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:03.158293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:03.199009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:03.200101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:03.200380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:03.200689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:03.228855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:03.239877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:03.240803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:03.242098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 196/381] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:03.247228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:03.253687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:03.260465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3fbb5-c5c8-4225-ba14-ee47a54cd17c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:03.303582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:03.304973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:03.305323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:03.305552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] PATH_INFO: `/v3/groups/9d7c2936e76442018fa2766efdde9306` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:03.312479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] RBAC: Authorizing `identity:get_group(group_id=9d7c2936e76442018fa2766efdde9306)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:03.313983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:03.316370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 186/382] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:03 2022] GET /identity/v3/groups/9d7c2936e76442018fa2766efdde9306 => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:03.322312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:03.481944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cd668-8b50-4dd5-8f91-02ebe800dc8a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:03.508570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae978d0a-a3de-4073-a26b-39cbe842ed5c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:03.556538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:03.557488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:03.557918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:03.558378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:03.559383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] RBAC: Adding query filter params (group.id=9d7c2936e76442018fa2766efdde9306, role.id=e1e5746208514068acadbab89907c473, scope.project.id=d39d97f2faca4faba12c96da090b5017) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:03.559858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:03.561488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:03.564994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:03.566325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:03.566613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:03.566892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:03.567871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:03.567974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:03.569647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:03.571710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 187/383] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:03 2022] GET /identity/v3/role_assignments?group.id=9d7c2936e76442018fa2766efdde9306&scope.project.id=d39d97f2faca4faba12c96da090b5017&role.id=e1e5746208514068acadbab89907c473 => generated 260 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:03.594245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 197/384] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:53:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:03.601066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:03.645906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:03.771468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e4b476-e0d6-4961-85de-a3bff09fe34c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:03.825094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:03.825999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:03.826282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:03.827751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:03.828731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:03.833292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:03.833292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 188/385] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:03 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:03.840341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:03.898327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:03.979016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdb9dd-0ffa-46c1-9784-f95f1531d290 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:04.028875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2834e-eafa-4a33-9284-15f170ba186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:04.029704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2834e-eafa-4a33-9284-15f170ba186d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:04.030054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2834e-eafa-4a33-9284-15f170ba186d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:04.030337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2834e-eafa-4a33-9284-15f170ba186d None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:04.031591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2834e-eafa-4a33-9284-15f170ba186d None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:04.033071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c2834e-eafa-4a33-9284-15f170ba186d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:04.035615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 198/386] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:03 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:04.041293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:04.210145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:04.222650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d371c17-39d0-4b2d-8600-790448847d60 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:04.270006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c35c0d-dd79-46b6-a704-91f828fca195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:04.270936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c35c0d-dd79-46b6-a704-91f828fca195 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:04.271224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c35c0d-dd79-46b6-a704-91f828fca195 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:04.271503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c35c0d-dd79-46b6-a704-91f828fca195 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:04.273715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c35c0d-dd79-46b6-a704-91f828fca195 None admin] RBAC: Authorizing `identity:create_grant(project_id=638fa7b1962d4ab7bf021ee0225dd847, group_id=3349b0a013bd408eac4d7b769d48ad6a, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:04.275772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c35c0d-dd79-46b6-a704-91f828fca195 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:04.332090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 189/387] 10.222.0.35 () {58 vars in 1442 bytes} [Tue May 24 09:53:04 2022] PUT /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:04.621143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92232-5990-4e85-9fbf-3d7e684fdb85 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:04.622163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92232-5990-4e85-9fbf-3d7e684fdb85 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:04.622653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92232-5990-4e85-9fbf-3d7e684fdb85 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:04.624523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 199/388] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:04.629816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ce6d-017c-4921-82a9-5ee1ea18929e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:04.630389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ce6d-017c-4921-82a9-5ee1ea18929e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:04.630892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ce6d-017c-4921-82a9-5ee1ea18929e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:04.658340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5587ce6d-017c-4921-82a9-5ee1ea18929e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:04.660410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5587ce6d-017c-4921-82a9-5ee1ea18929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:04.755982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 190/389] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:04.791197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92232-5990-4e85-9fbf-3d7e684fdb85 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:04.949957 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92232-5990-4e85-9fbf-3d7e684fdb85 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:04.963269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:04.994890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:04.995699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:04.995985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:04.996263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.015343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:05.024300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:05.025207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:05.026035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 200/390] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:05.031449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587ce6d-017c-4921-82a9-5ee1ea18929e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:05.134487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:05.165007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587ce6d-017c-4921-82a9-5ee1ea18929e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:05.212829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:05.213453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.213739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.214033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.214797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] RBAC: Adding query filter params (name=member) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:05.218219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:05.218219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:05.228822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 191/391] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:53:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:05.234932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:05.390214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eeeca9-f291-45d4-bf30-9edab40389e3 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:05.444651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15688520-c0f7-492d-b2c7-453e7a742058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:05.445841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15688520-c0f7-492d-b2c7-453e7a742058 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.446303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15688520-c0f7-492d-b2c7-453e7a742058 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.446754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15688520-c0f7-492d-b2c7-453e7a742058 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.448108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15688520-c0f7-492d-b2c7-453e7a742058 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:05.449796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15688520-c0f7-492d-b2c7-453e7a742058 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:05.451910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 201/392] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:05 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:05.458003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:05.540284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d71c0-dc35-40cb-8c88-297e3aa84e04 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.540836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d71c0-dc35-40cb-8c88-297e3aa84e04 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.541380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d71c0-dc35-40cb-8c88-297e3aa84e04 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.542913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 202/393] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:05.548308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075da548-a3a2-4618-9487-aa63d78c3570 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.548808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075da548-a3a2-4618-9487-aa63d78c3570 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.549247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075da548-a3a2-4618-9487-aa63d78c3570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.573457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-075da548-a3a2-4618-9487-aa63d78c3570 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:05.576184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-075da548-a3a2-4618-9487-aa63d78c3570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:05.633610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:05.669722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7b2ba-910b-4e09-a1d8-f49401936706 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:05.714207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 203/394] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:05.728022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:05.729235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.729754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.730233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.731678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:05.733379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:05.735682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 192/395] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:05 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:05.742378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075da548-a3a2-4618-9487-aa63d78c3570 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:05.758938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:05.785849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:05.881091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075da548-a3a2-4618-9487-aa63d78c3570 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:05.912439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3183c7-7bf6-49d4-9507-fbc88f713e71 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:05.917240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899111f5-3ab2-4e19-a606-00e118b16c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:05.918157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899111f5-3ab2-4e19-a606-00e118b16c67 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.918443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899111f5-3ab2-4e19-a606-00e118b16c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.918721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899111f5-3ab2-4e19-a606-00e118b16c67 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.920725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899111f5-3ab2-4e19-a606-00e118b16c67 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7757ba61ab423fa983d6fa71116119, group_id=3349b0a013bd408eac4d7b769d48ad6a, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:05.922207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899111f5-3ab2-4e19-a606-00e118b16c67 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:05.947572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f852e-bbb2-4f2c-a41b-06413621135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:05.948381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f852e-bbb2-4f2c-a41b-06413621135e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.948664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f852e-bbb2-4f2c-a41b-06413621135e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.948934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f852e-bbb2-4f2c-a41b-06413621135e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.960659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f852e-bbb2-4f2c-a41b-06413621135e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-584f852e-bbb2-4f2c-a41b-06413621135e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:05.968766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:05.969781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:05.970686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 193/396] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:05.972436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05594f-2946-46dc-ac27-02d6a65d63d7 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.972436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05594f-2946-46dc-ac27-02d6a65d63d7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.972436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05594f-2946-46dc-ac27-02d6a65d63d7 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:05.977529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 194/397] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:05.978569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05594f-2946-46dc-ac27-02d6a65d63d7 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:05.995067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 204/398] 10.222.0.35 () {58 vars in 1442 bytes} [Tue May 24 09:53:05 2022] PUT /identity/v3/projects/db7757ba61ab423fa983d6fa71116119/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:05.996907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dff0b-08d9-4b96-af57-62983088fa0d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:05.997483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dff0b-08d9-4b96-af57-62983088fa0d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:05.997943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dff0b-08d9-4b96-af57-62983088fa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.037293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-560dff0b-08d9-4b96-af57-62983088fa0d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:06.039518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-560dff0b-08d9-4b96-af57-62983088fa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:06.088126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:06.105784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:06.152314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05594f-2946-46dc-ac27-02d6a65d63d7 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:06.213299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 205/399] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:06.223754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:06.224433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.224772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.225102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.226065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:06.226433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:06.227827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:06.245018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dff0b-08d9-4b96-af57-62983088fa0d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:06.255890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 195/400] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:06.265309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:06.443457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dff0b-08d9-4b96-af57-62983088fa0d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:06.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e03508-ea80-4722-8666-b55decc93e58 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:06.491955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:06.502299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:06.503919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.508436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.508436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.509180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:06.510664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:06.515731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 196/401] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:06 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:06.520714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:06.521834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.522220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:06.522644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.523298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.536024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:06.544673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:06.545579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:06.546485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:06.549341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 206/402] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:06.557300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:06.576443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:06.677747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ef579-9474-437a-be9c-7ca7cb4648de None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:06.703379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e713-ad77-4211-bdbf-d39355a3c88d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:06.718194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d64712-ba34-46ae-9945-1a14e21e782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:06.719072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d64712-ba34-46ae-9945-1a14e21e782a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.719356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d64712-ba34-46ae-9945-1a14e21e782a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.719631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d64712-ba34-46ae-9945-1a14e21e782a None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.720891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d64712-ba34-46ae-9945-1a14e21e782a None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:06.723288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d64712-ba34-46ae-9945-1a14e21e782a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:06.725175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 197/403] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:06 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:06.726514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795bb5-b0e2-4ee7-9a5c-5d3a47967f52 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.726802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795bb5-b0e2-4ee7-9a5c-5d3a47967f52 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.727075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12795bb5-b0e2-4ee7-9a5c-5d3a47967f52 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.728651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 198/404] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:06.733259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12795bb5-b0e2-4ee7-9a5c-5d3a47967f52 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:06.744359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464b2ede-4170-46a8-a328-0bde0a40844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:06.745423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2ede-4170-46a8-a328-0bde0a40844b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.745706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2ede-4170-46a8-a328-0bde0a40844b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.745984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464b2ede-4170-46a8-a328-0bde0a40844b None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.747200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b2ede-4170-46a8-a328-0bde0a40844b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:06.747636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b2ede-4170-46a8-a328-0bde0a40844b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:06.749198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464b2ede-4170-46a8-a328-0bde0a40844b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:06.761542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 207/405] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:06.762927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732a7f-b675-4c5e-b011-3176e9ba932e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.763219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732a7f-b675-4c5e-b011-3176e9ba932e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.763505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52732a7f-b675-4c5e-b011-3176e9ba932e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.789503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-52732a7f-b675-4c5e-b011-3176e9ba932e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:06.790731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-52732a7f-b675-4c5e-b011-3176e9ba932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:06.841760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:06.864975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 208/406] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:06.865913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12795bb5-b0e2-4ee7-9a5c-5d3a47967f52 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:06.866101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732a7f-b675-4c5e-b011-3176e9ba932e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:06.898029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:06.911654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:06.912980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:06.913297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:06.913589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:06.916766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] RBAC: Authorizing `identity:create_grant(project_id=638fa7b1962d4ab7bf021ee0225dd847, group_id=3349b0a013bd408eac4d7b769d48ad6a, role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:06.918799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:06.995524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52732a7f-b675-4c5e-b011-3176e9ba932e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.017551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 199/407] 10.222.0.35 () {58 vars in 1442 bytes} [Tue May 24 09:53:06 2022] PUT /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/60347b7e855240eb9c04a1cb202a3500 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:07.019275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.041015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebdcee7-bc42-4211-8be8-2be0004d84e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.041941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdcee7-bc42-4211-8be8-2be0004d84e9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.042221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdcee7-bc42-4211-8be8-2be0004d84e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.042497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdcee7-bc42-4211-8be8-2be0004d84e9 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.043777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdcee7-bc42-4211-8be8-2be0004d84e9 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:07.045337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebdcee7-bc42-4211-8be8-2be0004d84e9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:07.051984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 209/408] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:06 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:07.053612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c70ed-67b7-4cb0-9c08-04f79d18aed0 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.053954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c70ed-67b7-4cb0-9c08-04f79d18aed0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.054233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4c70ed-67b7-4cb0-9c08-04f79d18aed0 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.059928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 210/409] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:07.059928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c70ed-67b7-4cb0-9c08-04f79d18aed0 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.149043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:07.162012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb306904-9022-4415-b3b8-612a91e76eab None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.177669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4c70ed-67b7-4cb0-9c08-04f79d18aed0 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.203880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:07.214344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.214875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.215159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.215432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.220335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db6d37-3d10-4525-b3b5-acaa569b0e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.221883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6d37-3d10-4525-b3b5-acaa569b0e94 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.222449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6d37-3d10-4525-b3b5-acaa569b0e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.223009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db6d37-3d10-4525-b3b5-acaa569b0e94 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.226024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db6d37-3d10-4525-b3b5-acaa569b0e94 None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:07.227571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db6d37-3d10-4525-b3b5-acaa569b0e94 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:07.229634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 211/410] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:07.230833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51bce1-fd5b-4704-9a9e-efebc255be87 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.231134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51bce1-fd5b-4704-9a9e-efebc255be87 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.231694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf51bce1-fd5b-4704-9a9e-efebc255be87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.236871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:07.244970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:07.245954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:07.246776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 200/411] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:07.247488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.253454 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-cf51bce1-fd5b-4704-9a9e-efebc255be87 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:07.253973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cf51bce1-fd5b-4704-9a9e-efebc255be87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:07.333333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 212/412] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:07.334061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51bce1-fd5b-4704-9a9e-efebc255be87 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.382157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8dbf59-6072-4a3a-99f0-479219b41f59 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.421622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:07.427165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.428651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.429137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.429640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.432786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] RBAC: Authorizing `identity:create_grant(project_id=db7757ba61ab423fa983d6fa71116119, group_id=3349b0a013bd408eac4d7b769d48ad6a, role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:07.436364 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:07.444790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf51bce1-fd5b-4704-9a9e-efebc255be87 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.480832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.481539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.481866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.482142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.482885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:07.483189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:07.484574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:07.494582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 213/413] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:07.495416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.515763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 201/414] 10.222.0.35 () {58 vars in 1442 bytes} [Tue May 24 09:53:07 2022] PUT /identity/v3/projects/db7757ba61ab423fa983d6fa71116119/groups/3349b0a013bd408eac4d7b769d48ad6a/roles/60347b7e855240eb9c04a1cb202a3500 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:07.522561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.621658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:07.682963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830fb2a-f20b-4e23-bcf9-79558413336d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.722347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be88675-2183-416f-87a6-0e671da3e947 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.741731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.742542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.742824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.743098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.746038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:07.762109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:07.774968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:07.775872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:07.776692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:07.779516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 214/415] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:07.785698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.786322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.786869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.787239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.787612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] PATH_INFO: `/v3/groups/9d7c2936e76442018fa2766efdde9306` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.788916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] RBAC: Authorizing `identity:get_group(group_id=9d7c2936e76442018fa2766efdde9306)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:07.790539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:07.792611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 202/416] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/groups/9d7c2936e76442018fa2766efdde9306 => generated 229 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:07.797671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.899364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f104bd-dddf-496b-8627-f03b69d24b11 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.912319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f858bed-5726-4392-a399-cd8a0937eeb2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:07.935534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.936508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.936962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.937466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.938394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:07.938890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:07.940494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:07.945647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:07.950095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 215/417] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:07.955371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:07.975744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:07.979924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:07.980227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:07.980515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] PATH_INFO: `/v3/projects/d39d97f2faca4faba12c96da090b5017` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:07.983300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] RBAC: Authorizing `identity:get_project(project_id=d39d97f2faca4faba12c96da090b5017)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:07.987255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:07.989776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 203/418] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/projects/d39d97f2faca4faba12c96da090b5017 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:07.994996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:08.081044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdab84d-8b1d-4973-aad0-dc8d92ccd86f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:08.082505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:08.120511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:08.121163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:08.121475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:08.121752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:08.200816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:08.202642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:08.281187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6739de-f5d4-449e-ba5f-e84546a21573 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:08.319396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:08.323905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:08.324805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:08.325086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:08.325387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] PATH_INFO: `/v3/projects/d39d97f2faca4faba12c96da090b5017/groups/9d7c2936e76442018fa2766efdde9306/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:08.333842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] RBAC: Authorizing `identity:create_grant(project_id=d39d97f2faca4faba12c96da090b5017, group_id=9d7c2936e76442018fa2766efdde9306, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:08.335223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:08.387573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 216/419] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:08.394765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:08.439325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 204/420] 10.222.0.35 () {58 vars in 1442 bytes} [Tue May 24 09:53:07 2022] PUT /identity/v3/projects/d39d97f2faca4faba12c96da090b5017/groups/9d7c2936e76442018fa2766efdde9306/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:53:08.517152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47d9e03-6a8d-4621-b940-cdd695383c77 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:08.557298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e263a3aa-0edc-4b63-b7b1-28fcdf976560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:08.558408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a3aa-0edc-4b63-b7b1-28fcdf976560 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:08.558691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a3aa-0edc-4b63-b7b1-28fcdf976560 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:08.558963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263a3aa-0edc-4b63-b7b1-28fcdf976560 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:08.560218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a3aa-0edc-4b63-b7b1-28fcdf976560 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:08.561859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e263a3aa-0edc-4b63-b7b1-28fcdf976560 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:08.563962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 217/421] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:08 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:08.569894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:08.631148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c108-42f6-4856-a653-5bfa66287117 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:08.631437 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c108-42f6-4856-a653-5bfa66287117 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:08.631823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe2c108-42f6-4856-a653-5bfa66287117 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:08.633640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 218/422] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:08.643895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24340e44-3b7b-4236-9096-7e30d4f1c247 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:08.644328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24340e44-3b7b-4236-9096-7e30d4f1c247 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:08.644648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24340e44-3b7b-4236-9096-7e30d4f1c247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:08.665873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-24340e44-3b7b-4236-9096-7e30d4f1c247 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:08.667082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-24340e44-3b7b-4236-9096-7e30d4f1c247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:08.668313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:08.692708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:08.747136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 219/423] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:08.759460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe76f339-7e00-4d05-b2e5-8ae5ab04d81f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:08.776613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24340e44-3b7b-4236-9096-7e30d4f1c247 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:08.816511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:08.817412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:08.817699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:08.817977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:08.819922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:08.821426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:08.825809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 205/424] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:08 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:08.832028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:08.956453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24340e44-3b7b-4236-9096-7e30d4f1c247 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:09.010622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448fd17-f567-4aa7-ad2b-5468b31c86ed None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:09.021323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:09.022139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.022418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.022693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.034543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:09.040709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:09.048550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:09.051180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:09.052048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 220/425] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:09.062004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:09.081453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:09.082286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.082572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.082897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.083749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:09.084064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:09.085526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:09.097814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 206/426] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:08 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 611 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:09.105699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:09.237421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163cd3d9-73d6-48a7-908a-d8fa7f4c6ce5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:09.287004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26774eb1-c3d5-4f8a-8b30-cb7066981765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:09.287701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774eb1-c3d5-4f8a-8b30-cb7066981765 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.287980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774eb1-c3d5-4f8a-8b30-cb7066981765 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.288257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26774eb1-c3d5-4f8a-8b30-cb7066981765 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.289024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26774eb1-c3d5-4f8a-8b30-cb7066981765 None admin] RBAC: Adding query filter params (name=member) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:09.289349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26774eb1-c3d5-4f8a-8b30-cb7066981765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:09.290745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26774eb1-c3d5-4f8a-8b30-cb7066981765 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:09.306177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 221/427] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:53:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:09.311678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:09.450679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066ed43-07ba-44c2-a8bd-5ec983eec812 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:09.487506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:09.488663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.489189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.489697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.538403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:09.567011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:09.568539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:09.608526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2821c-e1a4-408b-ae6b-f59e64e25df3 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.609143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2821c-e1a4-408b-ae6b-f59e64e25df3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.609681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e2821c-e1a4-408b-ae6b-f59e64e25df3 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.611247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 222/428] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:09.618020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbc597-d6c5-4ce5-9083-9499ba6cab7c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.618701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbc597-d6c5-4ce5-9083-9499ba6cab7c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.619060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbc597-d6c5-4ce5-9083-9499ba6cab7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.643789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9ffbc597-d6c5-4ce5-9083-9499ba6cab7c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:09.645336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9ffbc597-d6c5-4ce5-9083-9499ba6cab7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:09.700222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 207/429] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:09.706673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:09.741723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 223/430] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:09.765652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbc597-d6c5-4ce5-9083-9499ba6cab7c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:09.789626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:09.806456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:09.836298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff930032-d69c-449f-a78c-2fac12ecc8c1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:09.882638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:09.883543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.883823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.884096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.885296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:09.886786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:09.888774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 208/431] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:09 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:09.894782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:09.912765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbc597-d6c5-4ce5-9083-9499ba6cab7c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:09.961807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:09.962890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:09.963540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:09.964353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:09.980551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:09.991960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:09.993068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:09.994145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:09.997788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 224/432] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:10.004053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.028411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea3687-0824-460f-81f1-4ef390d0823b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.065591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.066694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.067001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.067373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.068733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:10.072187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:10.072513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 209/433] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:09 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:10.081429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.091784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:10.101735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:10.135244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afc39f3-b6eb-4ea9-b482-7d1db7672e59 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.187732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.188534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.188890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.189238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.190178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:10.190808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:10.192431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:10.202615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 225/434] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:10.209824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.236685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238126c9-a4fe-45f3-b789-6e5960381410 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.291543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f02ee-d017-46a5-a6f5-52c2e8e35ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.292867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f02ee-d017-46a5-a6f5-52c2e8e35ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.293390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f02ee-d017-46a5-a6f5-52c2e8e35ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.294026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f02ee-d017-46a5-a6f5-52c2e8e35ec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.295827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f02ee-d017-46a5-a6f5-52c2e8e35ec1 None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=af0d03236dab45f3a150b6281f50bb80, scope.project.id=db7757ba61ab423fa983d6fa71116119) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:10.296387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f02ee-d017-46a5-a6f5-52c2e8e35ec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:10.298711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f02ee-d017-46a5-a6f5-52c2e8e35ec1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:10.313514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 210/435] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=af0d03236dab45f3a150b6281f50bb80 => generated 611 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:10.352697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb905681-e29e-4862-8b4c-2c102cca562e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.385144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:10.392021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.392694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.392979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.393433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.473043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:10.474699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:10.494361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c55e-2f53-48f8-b1d3-b499f5ae01a7 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.494636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c55e-2f53-48f8-b1d3-b499f5ae01a7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.494921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5c55e-2f53-48f8-b1d3-b499f5ae01a7 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.496549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 211/436] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:10.503473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abb897-3ad4-4725-9dd3-4895dde93e68 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.503769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abb897-3ad4-4725-9dd3-4895dde93e68 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.504050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abb897-3ad4-4725-9dd3-4895dde93e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.527834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-73abb897-3ad4-4725-9dd3-4895dde93e68 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:10.530089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-73abb897-3ad4-4725-9dd3-4895dde93e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:10.605607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 226/437] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:53:10.613144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.619703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 212/438] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:10.644866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abb897-3ad4-4725-9dd3-4895dde93e68 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.649595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:10.653808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:10.727935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e305bfe-24e7-4b00-a5f9-ec3d5ab93473 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.757572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abb897-3ad4-4725-9dd3-4895dde93e68 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.765050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.766036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.766809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.766886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] PATH_INFO: `/v3/projects/638fa7b1962d4ab7bf021ee0225dd847` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.768029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] RBAC: Authorizing `identity:get_project(project_id=638fa7b1962d4ab7bf021ee0225dd847)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:10.770075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:10.776580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 227/439] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/projects/638fa7b1962d4ab7bf021ee0225dd847 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:10.777398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.799851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.801073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.801385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.801676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.811855 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:10.820080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:10.820961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:10.821882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 213/440] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:10.826910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.885843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:10.889488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a4cf31-be79-4bdb-ac9e-1d1e81beed89 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.894898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:10.924615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83facb74-580f-46b3-bec1-1de0ea182959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.925683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83facb74-580f-46b3-bec1-1de0ea182959 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.925972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83facb74-580f-46b3-bec1-1de0ea182959 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.926244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83facb74-580f-46b3-bec1-1de0ea182959 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.927434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83facb74-580f-46b3-bec1-1de0ea182959 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:10.928711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf13724-1a8b-4595-92fa-055e79a0b1a4 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:10.928920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83facb74-580f-46b3-bec1-1de0ea182959 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:10.930569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 228/441] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:10.931921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944cdf6-0555-4d93-ae16-ee3e13f20c37 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.932204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944cdf6-0555-4d93-ae16-ee3e13f20c37 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.932482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7944cdf6-0555-4d93-ae16-ee3e13f20c37 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.933872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 229/442] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:10.936343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944cdf6-0555-4d93-ae16-ee3e13f20c37 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:10.968874 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a0ee0-ece0-4dac-be4e-dc60bc75ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:10.969590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0ee0-ece0-4dac-be4e-dc60bc75ce67 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.969878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0ee0-ece0-4dac-be4e-dc60bc75ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.970158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a0ee0-ece0-4dac-be4e-dc60bc75ce67 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.970909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a0ee0-ece0-4dac-be4e-dc60bc75ce67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:10.971222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a0ee0-ece0-4dac-be4e-dc60bc75ce67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:10.973363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a0ee0-ece0-4dac-be4e-dc60bc75ce67 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:10.981184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 214/443] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:10.982467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59b2a-b668-4201-a7da-f1bc07fa2207 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:10.982758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59b2a-b668-4201-a7da-f1bc07fa2207 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:10.983037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc59b2a-b668-4201-a7da-f1bc07fa2207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:10.999117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0bc59b2a-b668-4201-a7da-f1bc07fa2207 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:11.000276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0bc59b2a-b668-4201-a7da-f1bc07fa2207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:11.041232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7944cdf6-0555-4d93-ae16-ee3e13f20c37 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.066611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 215/444] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:11.067447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59b2a-b668-4201-a7da-f1bc07fa2207 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.084067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.084733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.085018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.085320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.086167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=638fa7b1962d4ab7bf021ee0225dd847) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:11.086480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.087940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.094089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 230/445] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:10 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=638fa7b1962d4ab7bf021ee0225dd847&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:11.094854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.134567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.138010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.190730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc59b2a-b668-4201-a7da-f1bc07fa2207 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.221065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fb7da-e0a7-499b-bac4-09e9ace294f0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.229623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.230300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.230586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.230867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.263547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.264836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.265393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.265855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.278935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:11.287175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:11.288097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:11.289344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 231/446] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:11.294220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.311596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.313155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.382511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.408323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.420768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f37c2-cae6-4634-b9ca-4cb558f7a66e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.447136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 216/447] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:11.452309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.459774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.460461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.460747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.461028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.461831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:11.462142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.463537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.475054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 232/448] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:11.481827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.572452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4756c2a-fd29-4ca4-a5e1-052725bc9f33 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.600987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9d5d7-d111-4d0a-8015-28ff572a9762 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.617714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.618584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.618866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.619143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] PATH_INFO: `/v3/projects/db7757ba61ab423fa983d6fa71116119` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.620334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] RBAC: Authorizing `identity:get_project(project_id=db7757ba61ab423fa983d6fa71116119)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.621825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.623514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 217/449] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/projects/db7757ba61ab423fa983d6fa71116119 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:11.629158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.642415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.643171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.644091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.644191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.664543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.669303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.715825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.717404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.739896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a925286-9d5a-4676-9845-d76bb4fabb7c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.772112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.772958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.773235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.773542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] PATH_INFO: `/v3/groups/3349b0a013bd408eac4d7b769d48ad6a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.774703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] RBAC: Authorizing `identity:get_group(group_id=3349b0a013bd408eac4d7b769d48ad6a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.776124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.777757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 218/450] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/groups/3349b0a013bd408eac4d7b769d48ad6a => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:11.783483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.818822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 233/451] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:11.824242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:11.874251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.877510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:11.880826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2620a7d-ae5b-4af6-8765-bcfd884412a5 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.914996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.915612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.915894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.916167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.917028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] RBAC: Adding query filter params (group.id=3349b0a013bd408eac4d7b769d48ad6a, role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=db7757ba61ab423fa983d6fa71116119) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:11.917496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.918927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.922995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3718524b-16b8-4da1-893c-6e45d0eeca2f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:11.924960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 219/452] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/role_assignments?group.id=3349b0a013bd408eac4d7b769d48ad6a&scope.project.id=db7757ba61ab423fa983d6fa71116119&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:11.959518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:11.960404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:11.960687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:11.960960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] PATH_INFO: `/v3/projects/d39d97f2faca4faba12c96da090b5017` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:11.962234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] RBAC: Authorizing `identity:get_project(project_id=d39d97f2faca4faba12c96da090b5017)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:11.963732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:11.965511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 234/453] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/projects/d39d97f2faca4faba12c96da090b5017 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:11.971181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:12.078537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c20cf1-647c-4ca8-a8e2-2031663cb03b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:12.114349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639bbabc-2587-4b2d-a945-16e05f7ef9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:12.115206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bbabc-2587-4b2d-a945-16e05f7ef9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:12.115490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bbabc-2587-4b2d-a945-16e05f7ef9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:12.115770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639bbabc-2587-4b2d-a945-16e05f7ef9c5 None admin] PATH_INFO: `/v3/groups/9d7c2936e76442018fa2766efdde9306` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:12.116945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bbabc-2587-4b2d-a945-16e05f7ef9c5 None admin] RBAC: Authorizing `identity:get_group(group_id=9d7c2936e76442018fa2766efdde9306)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:12.118407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639bbabc-2587-4b2d-a945-16e05f7ef9c5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:12.120013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 220/454] 10.222.0.35 () {56 vars in 1180 bytes} [Tue May 24 09:53:11 2022] GET /identity/v3/groups/9d7c2936e76442018fa2766efdde9306 => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:12.125786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:12.227735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f593f271-115f-4067-a6d0-f05727cd08d4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:12.247268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:12.260911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3b20ce-0f83-431c-86fa-daa1af6aea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:12.261590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b20ce-0f83-431c-86fa-daa1af6aea65 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:12.261868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b20ce-0f83-431c-86fa-daa1af6aea65 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:12.262142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3b20ce-0f83-431c-86fa-daa1af6aea65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:12.262984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b20ce-0f83-431c-86fa-daa1af6aea65 None admin] RBAC: Adding query filter params (group.id=9d7c2936e76442018fa2766efdde9306, role.id=e1e5746208514068acadbab89907c473, scope.project.id=d39d97f2faca4faba12c96da090b5017) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:12.263405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b20ce-0f83-431c-86fa-daa1af6aea65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:12.264849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3b20ce-0f83-431c-86fa-daa1af6aea65 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:12.271775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 235/455] 10.222.0.35 () {56 vars in 1376 bytes} [Tue May 24 09:53:12 2022] GET /identity/v3/role_assignments?group.id=9d7c2936e76442018fa2766efdde9306&scope.project.id=d39d97f2faca4faba12c96da090b5017&role.id=e1e5746208514068acadbab89907c473 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:14.966502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba1b15-28da-4a0f-a3cd-687940a6d99e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:14.967039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba1b15-28da-4a0f-a3cd-687940a6d99e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:14.967039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba1b15-28da-4a0f-a3cd-687940a6d99e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:14.968447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 221/456] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:14.974778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341ecb0-4cb4-4067-9d55-34c8d1824fd4 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:14.975282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341ecb0-4cb4-4067-9d55-34c8d1824fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:14.975358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341ecb0-4cb4-4067-9d55-34c8d1824fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:14.994449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9341ecb0-4cb4-4067-9d55-34c8d1824fd4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:14.995627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9341ecb0-4cb4-4067-9d55-34c8d1824fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:15.082628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 236/457] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:15.131950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba1b15-28da-4a0f-a3cd-687940a6d99e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:15.180596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:15.192971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b2780-a654-4b42-b002-303627da4a62 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.193287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b2780-a654-4b42-b002-303627da4a62 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.193555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b2780-a654-4b42-b002-303627da4a62 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.195265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 237/458] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:15.200279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b3400-3186-4a30-95d5-7ce03769b9ce None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.200820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b3400-3186-4a30-95d5-7ce03769b9ce None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.201368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b3400-3186-4a30-95d5-7ce03769b9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.245992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c47b3400-3186-4a30-95d5-7ce03769b9ce None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:15.245992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c47b3400-3186-4a30-95d5-7ce03769b9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:15.276876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba1b15-28da-4a0f-a3cd-687940a6d99e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:15.336868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:15.337707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.337995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.338268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.359481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:15.360941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:15.377633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 238/459] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:15.397035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:15.397968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:15.398821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:15.398821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:15.398821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:15.398821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:15.398821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 222/460] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:15.404090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b3400-3186-4a30-95d5-7ce03769b9ce None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:15.406505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:15.497716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:15.515465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:15.517192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03f8fe-f115-49fc-a275-3d6ce3e40c58 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:15.552862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256be0b0-1446-4226-9300-47e6bbf2fd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:15.554086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256be0b0-1446-4226-9300-47e6bbf2fd4b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.554577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256be0b0-1446-4226-9300-47e6bbf2fd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.555054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256be0b0-1446-4226-9300-47e6bbf2fd4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.567069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256be0b0-1446-4226-9300-47e6bbf2fd4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:15.568986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256be0b0-1446-4226-9300-47e6bbf2fd4b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-256be0b0-1446-4226-9300-47e6bbf2fd4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:15.588958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:15.589896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:15.590827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:15.590827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:15.590827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:15.590827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:15.593190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 223/461] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:15.599733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa41b7-371d-4800-8dc4-667bfd8988d5 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.600389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa41b7-371d-4800-8dc4-667bfd8988d5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.600969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa41b7-371d-4800-8dc4-667bfd8988d5 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.602864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 224/462] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:15.603778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa41b7-371d-4800-8dc4-667bfd8988d5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:15.613918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b3400-3186-4a30-95d5-7ce03769b9ce None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:15.651509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02081cd1-3a42-40a8-ada3-fc8c2910ca36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:15.652174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02081cd1-3a42-40a8-ada3-fc8c2910ca36 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.652453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02081cd1-3a42-40a8-ada3-fc8c2910ca36 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.652726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02081cd1-3a42-40a8-ada3-fc8c2910ca36 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.653589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02081cd1-3a42-40a8-ada3-fc8c2910ca36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:15.653893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02081cd1-3a42-40a8-ada3-fc8c2910ca36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:15.655290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02081cd1-3a42-40a8-ada3-fc8c2910ca36 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:15.674202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 239/463] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:15.675839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867961-91ad-43b9-8c2b-4bcf3ce16760 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.676129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867961-91ad-43b9-8c2b-4bcf3ce16760 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.676408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71867961-91ad-43b9-8c2b-4bcf3ce16760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.694179 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-71867961-91ad-43b9-8c2b-4bcf3ce16760 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:15.695356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-71867961-91ad-43b9-8c2b-4bcf3ce16760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:15.719347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa41b7-371d-4800-8dc4-667bfd8988d5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:15.755655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:15.761166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be65d4-e95f-41b3-aa72-09fd0353a117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:15.776026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be65d4-e95f-41b3-aa72-09fd0353a117 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.776026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be65d4-e95f-41b3-aa72-09fd0353a117 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.776026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be65d4-e95f-41b3-aa72-09fd0353a117 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.776026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be65d4-e95f-41b3-aa72-09fd0353a117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:15.776026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be65d4-e95f-41b3-aa72-09fd0353a117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:15.776026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be65d4-e95f-41b3-aa72-09fd0353a117 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:15.776449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 240/464] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:15.776449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867961-91ad-43b9-8c2b-4bcf3ce16760 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:15.787293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 225/465] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:15.788803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21facad-de07-4842-8f0f-4b91b70385e6 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.789295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21facad-de07-4842-8f0f-4b91b70385e6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.789722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21facad-de07-4842-8f0f-4b91b70385e6 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.795208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 226/466] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:15.796134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21facad-de07-4842-8f0f-4b91b70385e6 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:15.818784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:15.888309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71867961-91ad-43b9-8c2b-4bcf3ce16760 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:15.907938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21facad-de07-4842-8f0f-4b91b70385e6 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:15.924583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac239f6-a50d-4c06-9469-a2471748e9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:15.925561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac239f6-a50d-4c06-9469-a2471748e9c4 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.925817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac239f6-a50d-4c06-9469-a2471748e9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.926092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac239f6-a50d-4c06-9469-a2471748e9c4 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.926805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac239f6-a50d-4c06-9469-a2471748e9c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:15.928267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac239f6-a50d-4c06-9469-a2471748e9c4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:15.942369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:15.943036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:15.943320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:15.943597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:15.944298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:15.945758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:16.009993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 241/467] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:53:15 2022] POST /identity/v3/users => generated 246 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:16.011258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776ba03-c751-4f1c-8350-ef43323de292 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.011555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776ba03-c751-4f1c-8350-ef43323de292 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.011819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776ba03-c751-4f1c-8350-ef43323de292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.032482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9776ba03-c751-4f1c-8350-ef43323de292 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:16.033868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9776ba03-c751-4f1c-8350-ef43323de292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:16.041382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 227/468] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:53:15 2022] POST /identity/v3/users => generated 248 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:16.042940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:16.113698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:16.191786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 242/469] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:16.214287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11916d-8bfa-416f-843d-872134b76e8f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:16.230451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776ba03-c751-4f1c-8350-ef43323de292 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:16.273811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:16.295718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:16.296534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.296816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.297091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.308939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:16.310479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:16.328594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:16.329549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:16.330424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:16.330424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:16.330424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:16.330424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:16.330424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 228/470] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:16.336837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:16.377013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776ba03-c751-4f1c-8350-ef43323de292 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:16.427734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5db496-0f32-416a-9183-2a9a4ede47c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:16.428541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5db496-0f32-416a-9183-2a9a4ede47c5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.428826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5db496-0f32-416a-9183-2a9a4ede47c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.429099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5db496-0f32-416a-9183-2a9a4ede47c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.441252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5db496-0f32-416a-9183-2a9a4ede47c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:16.442915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5db496-0f32-416a-9183-2a9a4ede47c5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:16.461109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-1b5db496-0f32-416a-9183-2a9a4ede47c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:16.463359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:16.464190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:16.465385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:16.465385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:16.465385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:16.465385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:16.465385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 243/471] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:16.466239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6ce1-6e24-43d7-81e0-643e09ef4f4b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.466539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6ce1-6e24-43d7-81e0-643e09ef4f4b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.466835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e6ce1-6e24-43d7-81e0-643e09ef4f4b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.468379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 244/472] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:16.469182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44651ff-011c-4b70-b1e6-0d8592f34062 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:16.471014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e6ce1-6e24-43d7-81e0-643e09ef4f4b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:16.506064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91ca8c-896a-4e84-8b5c-99ce01316388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:16.506737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ca8c-896a-4e84-8b5c-99ce01316388 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.507021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ca8c-896a-4e84-8b5c-99ce01316388 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.507297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91ca8c-896a-4e84-8b5c-99ce01316388 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.508044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ca8c-896a-4e84-8b5c-99ce01316388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:16.508350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ca8c-896a-4e84-8b5c-99ce01316388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:16.509778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91ca8c-896a-4e84-8b5c-99ce01316388 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:16.522901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 229/473] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:16.524245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b61fb-1902-4c07-a86c-634f37c8f4dd None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.524475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b61fb-1902-4c07-a86c-634f37c8f4dd None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.524757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b61fb-1902-4c07-a86c-634f37c8f4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.546627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-cd6b61fb-1902-4c07-a86c-634f37c8f4dd None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:16.547740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-cd6b61fb-1902-4c07-a86c-634f37c8f4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:16.557674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:16.586666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e6ce1-6e24-43d7-81e0-643e09ef4f4b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:16.618633 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 230/474] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:16.619552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b61fb-1902-4c07-a86c-634f37c8f4dd None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:16.624610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:16.625553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.626043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.626554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.627668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:16.628241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:16.629994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:16.643040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 245/475] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:16.650283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:16.714850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:16.727416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b61fb-1902-4c07-a86c-634f37c8f4dd None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:16.764007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd1bfc0-0c6a-4b99-815c-5f74cee61757 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:16.770860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:16.771544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.771829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.772107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.773418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:16.775405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:16.797225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:16.803211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02972161-9153-4261-b623-698e48d49f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:16.803962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02972161-9153-4261-b623-698e48d49f6e None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:16.804321 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02972161-9153-4261-b623-698e48d49f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:16.804688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02972161-9153-4261-b623-698e48d49f6e None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:16.805501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02972161-9153-4261-b623-698e48d49f6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:16.807035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02972161-9153-4261-b623-698e48d49f6e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:16.884663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 231/476] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:53:16 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:16.885508 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:16.909196 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 246/477] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:53:16 2022] POST /identity/v3/users => generated 249 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:17.022654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596ef2f-1855-481d-89bb-8814cc2c89f4 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:17.076840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:17.077678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:17.077856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:17.078005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:17.078767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:17.080183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:17.113529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.sql.core [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 09:53:17.113529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 09:53:17.113529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [parameters: {'id': 'ff8bb9c957204abf82f083c1dad29d33', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 09:53:17.113529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80404) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application result = self._query(query) May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application conn.query(q) May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application result.read() May 24 09:53:17.131393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return ref.to_dict() May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 09:53:17.133559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application next(self.gen) May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application yield resource May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application next(self.gen) May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self.session.rollback() May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self.force_reraise() May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise self.value May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application session.commit() May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 09:53:17.135758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._prepare_impl() May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self.session.flush() May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._flush(objects) May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application compat.raise_( May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application flush_context.execute() May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rec.execute(self) May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 09:53:17.137854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 09:53:17.139966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application result = self._query(query) May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application conn.query(q) May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application result.read() May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 09:53:17.142243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [parameters: {'id': 'ff8bb9c957204abf82f083c1dad29d33', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:17.149661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 24 09:53:17.151385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:17.153424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 232/478] 10.222.0.35 () {60 vars in 1131 bytes} [Tue May 24 09:53:16 2022] POST /identity/v3/roles => generated 144 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:17.153561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02972161-9153-4261-b623-698e48d49f6e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:17.275142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02972161-9153-4261-b623-698e48d49f6e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:17.316471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d970db-60fc-4c0e-9b81-c45785deacbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:17.317297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d970db-60fc-4c0e-9b81-c45785deacbc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:17.317608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d970db-60fc-4c0e-9b81-c45785deacbc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:17.317884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d970db-60fc-4c0e-9b81-c45785deacbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:17.329647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d970db-60fc-4c0e-9b81-c45785deacbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-52d970db-60fc-4c0e-9b81-c45785deacbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:17.338007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:17.339917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:17.340922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 247/479] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:17.345029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:17.358362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:17.467622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a0d51-6cba-40bc-a019-ddb5b21d8773 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:17.504214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8c973e-5669-4976-825c-4c9fb1625bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:17.504995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c973e-5669-4976-825c-4c9fb1625bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:17.505297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c973e-5669-4976-825c-4c9fb1625bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:17.505578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8c973e-5669-4976-825c-4c9fb1625bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:17.506338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c973e-5669-4976-825c-4c9fb1625bc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:17.506645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c973e-5669-4976-825c-4c9fb1625bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:17.508015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8c973e-5669-4976-825c-4c9fb1625bc6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:17.522159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 233/480] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:18.626523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f8220-5b2a-4410-ac20-d65509f2909b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:18.627617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f8220-5b2a-4410-ac20-d65509f2909b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:18.628907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f8220-5b2a-4410-ac20-d65509f2909b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:18.631000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 248/481] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:18.637066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a75615-a518-425f-a4ab-e5b4e4bcb96a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:18.638021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a75615-a518-425f-a4ab-e5b4e4bcb96a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:18.638595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a75615-a518-425f-a4ab-e5b4e4bcb96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:18.661362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b6a75615-a518-425f-a4ab-e5b4e4bcb96a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:18.662889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b6a75615-a518-425f-a4ab-e5b4e4bcb96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:18.673898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96730-4136-4f4e-b869-eccf9f8f2648 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:18.673898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96730-4136-4f4e-b869-eccf9f8f2648 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:18.673898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96730-4136-4f4e-b869-eccf9f8f2648 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:18.673898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 249/482] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:18.680334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea905d33-3b0c-49b2-b4e3-172aa30f2923 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:18.682989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea905d33-3b0c-49b2-b4e3-172aa30f2923 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:18.682989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea905d33-3b0c-49b2-b4e3-172aa30f2923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:18.699108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ea905d33-3b0c-49b2-b4e3-172aa30f2923 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:18.700598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ea905d33-3b0c-49b2-b4e3-172aa30f2923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:18.756692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:18.768965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 234/483] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:18.773001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:18.803863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 250/484] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:18.806296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a75615-a518-425f-a4ab-e5b4e4bcb96a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:18.827970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea905d33-3b0c-49b2-b4e3-172aa30f2923 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:18.967186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a75615-a518-425f-a4ab-e5b4e4bcb96a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:18.995181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea905d33-3b0c-49b2-b4e3-172aa30f2923 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.011791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.012876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.013365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.013820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.037591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:19.052971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:19.053919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:19.054726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 235/485] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:19.063398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:19.066859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.067893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.068389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.068854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.081823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:19.090874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:19.091778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:19.092584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:19.095306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 251/486] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:19.101145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:19.147204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:19.218388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e65e8d-460c-4644-8758-1a6c883b7a86 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.238706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:19.259601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.261023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.261633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.262061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.262967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:19.263453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:19.264978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:19.275990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 252/487] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:19.281586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:19.359215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500d53c-5dc5-4939-a8cc-530b1114ab70 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.399120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:19.403138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0bb95-1d63-41db-b65e-8eeb1d51f4e8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.441912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.443453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.443914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.444339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.456567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:19.458469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:19.470584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.471198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.472092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.472447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.473211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:19.473548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:19.474922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:19.481324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:19.482194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:19.483056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:19.483056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:19.483056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:19.483056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:19.484944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 253/488] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:19.490759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:19.496442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 236/489] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:19.504460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:19.664375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde745d-5466-400c-8f89-fdba895d6bb4 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.692233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:19.705298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.706972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.716278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.716278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.716448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd7bb5-2db1-491c-bcbf-f73766e5d05c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.718819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:19.720440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:19.744119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:19.745171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:19.746183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:19.746183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:19.746183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:19.746183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:19.748001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 237/490] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:19.753246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:19.783690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.786909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.787433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.787851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.788750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:19.790706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:19.792401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:19.821395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 254/491] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:19.824517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:19.855027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:19.918450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67be10-62b0-40fb-8de9-b6d95949c3ad None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.959506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f7017-e7c0-41cd-baf2-fa9a25a1d9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:19.960293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f7017-e7c0-41cd-baf2-fa9a25a1d9dd None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.960616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f7017-e7c0-41cd-baf2-fa9a25a1d9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.960901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f7017-e7c0-41cd-baf2-fa9a25a1d9dd None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.961791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f7017-e7c0-41cd-baf2-fa9a25a1d9dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:19.962027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f7017-e7c0-41cd-baf2-fa9a25a1d9dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:19.963409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f7017-e7c0-41cd-baf2-fa9a25a1d9dd None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:19.967203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc319a4-92b7-409c-b4aa-c784ecda8336 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:19.976098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 238/492] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:19.980033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ab25-ebc8-40d3-9b9f-178fcf961ed1 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.980033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ab25-ebc8-40d3-9b9f-178fcf961ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.980033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa10ab25-ebc8-40d3-9b9f-178fcf961ed1 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.981401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 239/493] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:19.983266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921ac3c-0bd9-4999-9b12-5f9bf78ddb98 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:19.983558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921ac3c-0bd9-4999-9b12-5f9bf78ddb98 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:19.983836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0921ac3c-0bd9-4999-9b12-5f9bf78ddb98 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:19.985333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 240/494] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:19.986034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921ac3c-0bd9-4999-9b12-5f9bf78ddb98 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.007941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8d6b4-7c07-43f9-aef6-b9ac45bdec30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.008666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8d6b4-7c07-43f9-aef6-b9ac45bdec30 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.008950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8d6b4-7c07-43f9-aef6-b9ac45bdec30 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.009225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8d6b4-7c07-43f9-aef6-b9ac45bdec30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.017257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8d6b4-7c07-43f9-aef6-b9ac45bdec30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:20.019307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8d6b4-7c07-43f9-aef6-b9ac45bdec30 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:20.028414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-0ff8d6b4-7c07-43f9-aef6-b9ac45bdec30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:20.034930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:20.035860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 255/495] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d48ce-fe58-4a39-8824-74440fd99f89 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.047870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d48ce-fe58-4a39-8824-74440fd99f89 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.048584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d48ce-fe58-4a39-8824-74440fd99f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.054459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f39d48ce-fe58-4a39-8824-74440fd99f89 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:20.055847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f39d48ce-fe58-4a39-8824-74440fd99f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:20.109567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0921ac3c-0bd9-4999-9b12-5f9bf78ddb98 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.121488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 256/496] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:20.123030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a760b-583a-418e-8c46-55c42e9c4233 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.123315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a760b-583a-418e-8c46-55c42e9c4233 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.123590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a760b-583a-418e-8c46-55c42e9c4233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.128764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.150336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-381a760b-583a-418e-8c46-55c42e9c4233 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:20.151618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-381a760b-583a-418e-8c46-55c42e9c4233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:20.159675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.160602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.161013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.161413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.171164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:20.173304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:20.191926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:20.192854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:20.193813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:20.195716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 241/497] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:53:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:20.196640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.226646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 257/498] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:20.227784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a760b-583a-418e-8c46-55c42e9c4233 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.305898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.313161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56a4d3-84ad-4c5a-b18b-d308bca2467b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.346750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a760b-583a-418e-8c46-55c42e9c4233 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.350261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.350936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.351219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.351495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.352338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:20.352645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:20.354135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:20.379490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.385222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.386041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.386318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.390651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.398178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 242/499] 10.222.0.35 () {56 vars in 1137 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:20.398178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.402589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:20.410660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:20.411599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:20.412433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 258/500] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:20.413113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.504510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0675c9-277f-44e1-a135-9abafc808f6b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.514908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1c07e-33bc-432c-b880-5342cca8c58d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.538937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.539607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.539887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.540165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.541000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:20.541347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:20.542777 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:20.549805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.550827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.551113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.551386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.559683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 243/501] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:20.560415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.561896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:20.570022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:20.571013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:20.571839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 259/502] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:20.572494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.572801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.624753 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.662626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2ac8d-3d21-43ce-b8ce-7a4400eaed19 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.671652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa9123-8d7a-41ec-aeb0-857b1a1216a5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.696776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.697459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.697745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.698025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.705234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822b338-0cd9-431f-a8a0-73b44840dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.706053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b338-0cd9-431f-a8a0-73b44840dc6e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.706419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b338-0cd9-431f-a8a0-73b44840dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.706693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822b338-0cd9-431f-a8a0-73b44840dc6e None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.707432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822b338-0cd9-431f-a8a0-73b44840dc6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:20.707736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822b338-0cd9-431f-a8a0-73b44840dc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:20.709121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822b338-0cd9-431f-a8a0-73b44840dc6e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:20.716833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 260/503] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:20.718063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236430e8-3de8-4efd-8f94-011d4c398cec None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.718354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236430e8-3de8-4efd-8f94-011d4c398cec None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.718616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236430e8-3de8-4efd-8f94-011d4c398cec None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.719828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 261/504] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:20.720511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236430e8-3de8-4efd-8f94-011d4c398cec None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.768814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:20.770390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:20.781119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.820326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236430e8-3de8-4efd-8f94-011d4c398cec None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.833707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:20.854137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d61f82-a060-473c-a795-fa55c32fa6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:20.854783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d61f82-a060-473c-a795-fa55c32fa6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:20.855068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d61f82-a060-473c-a795-fa55c32fa6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:20.855342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d61f82-a060-473c-a795-fa55c32fa6f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:20.870424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 244/505] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:20.871259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:20.930397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d61f82-a060-473c-a795-fa55c32fa6f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:20.931871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d61f82-a060-473c-a795-fa55c32fa6f2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:20.967943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ef4a1-7060-4537-b990-cfa0621f8e5e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:20.979246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.003580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.004053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.004554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.005109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.005900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:21.006220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.007588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.015714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 245/506] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:21.016336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.033303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 262/507] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:21.033303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202d253-34dc-47bc-bb78-0601fe6cf524 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.033303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202d253-34dc-47bc-bb78-0601fe6cf524 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.033586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202d253-34dc-47bc-bb78-0601fe6cf524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.050567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7202d253-34dc-47bc-bb78-0601fe6cf524 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:21.051938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7202d253-34dc-47bc-bb78-0601fe6cf524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:21.061301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.117006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82afe6-1ede-4b24-95dc-e033a6e5dde3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.119263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 263/508] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:21.120083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202d253-34dc-47bc-bb78-0601fe6cf524 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.156449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.157247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.157556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.157831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.170781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.172237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:21.189610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:21.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:21.191848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:21.191848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:21.191848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.191848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:21.191848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 246/509] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:21.192767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.223209 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.242918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202d253-34dc-47bc-bb78-0601fe6cf524 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.276666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.277624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.277904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.278183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.288672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.290168 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.292660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.300302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169a39a0-6e9e-4a0c-99e1-cbe5f28d1629 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:21.307556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:21.308396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:21.309290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:21.309290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:21.309290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.309290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:21.309290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 264/510] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:21.309579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.337517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.338309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.338592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.338870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.348784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.350263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:21.369625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:21.370482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:21.371345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:21.371345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:21.371345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.371345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:21.371345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 247/511] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:21.371614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.413280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6692f28-d73c-4850-9025-e0bf7294bcf7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.446478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.447259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.447539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.447760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.447862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.458495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.459941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.470794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e3bec-8432-4ead-acb0-6fefe0d7bb77 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:21.478043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:21.479081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:21.484752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:21.484752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:21.484752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.484752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:21.484752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 265/512] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:21.484752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.505962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.506754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.507043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.507317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.517357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.519359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.529427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:21.538411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:21.544818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:21.545704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:21.545704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:21.545704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:21.545704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:21.545704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 248/513] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:21.545704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.582807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcb9fc-574c-4659-a3bf-7f96adc8650d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.617288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.617980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.618258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.618533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.619255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:21.619555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.620970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.632339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 266/514] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:21.633229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.645043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb46479-fe92-4a5d-b6f9-c3be98db3851 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.671951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.679708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.680366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.680651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.680931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.681690 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:21.682001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.686822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.695758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 249/515] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:21.696496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.735920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d9ff-63a5-41f1-a1ad-e124a209346c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.744893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.769715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.770547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.770990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.771312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.772348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:21.772655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.774086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.786860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 267/516] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:21.787754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.803016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c547b-9b02-46fa-b1d0-11f8fe82ba92 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.836979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.837667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.837954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.838232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.838958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:21.839268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.840635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.852804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 250/517] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:21.853618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.895493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6ef277-8157-42f0-ad24-0ec4aff971c2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.905206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.930608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.931266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.931546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.931827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:21.932541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:21.932843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:21.934256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:21.946008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 268/518] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:21.947201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:21.957384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf53657-c148-48c4-9141-34b9478a7db3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:21.976601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:21.991782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:21.992573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:21.992855 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:21.993132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.000334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.001811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:22.016536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:22.017920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 251/519] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:22.019311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.048972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf3e199-1f82-4f18-b063-702de659a309 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.082784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.083931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.084268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.084549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.091634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.093083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:22.108815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:22.109731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 269/520] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:22.110714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.121222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3a817-b79d-4269-9e94-474f928cf33f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.130808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:22.155276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.156004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.156286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.156560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.165055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.167076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:22.182026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:22.182970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:22.183870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 252/521] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:22.184522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.206700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:22.218291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c754e86-65a0-474c-88d6-72c76c3c1675 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.251817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.252600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.253070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.253383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.259922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.261436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:22.275736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:22.276776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:22.277674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 270/522] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:22.278172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.288418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce879b-e76b-49e8-b24a-3cd3c63afd8d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.321479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.322147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.322439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.322714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.323416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.324852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.381207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915444d9-14f5-4b9f-b4e0-418314a070e0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.410161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 253/523] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:53:22 2022] POST /identity/v3/users => generated 247 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:22.410917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.418950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.419650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.419979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.420205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.421790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:22.422103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.424044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.440940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:22.474959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 271/524] 10.222.0.35 () {56 vars in 1137 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:22.474959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.481140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:22.528918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f31d3-ca43-48bb-83de-ed231160b775 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.569676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.570357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.570643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.570915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.571725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:22.572035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.573509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.591786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa05f48-c03b-491d-a35f-624ddad8ade5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.598230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 254/525] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:22.598230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.643135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.644845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.645504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.645769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.646775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:22.647088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.648948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.668068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 272/526] 10.222.0.35 () {56 vars in 1139 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:22.668645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.694709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:22.712599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb942371-2ae4-4162-9b90-3e255f7156a2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.734238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:22.751821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.752486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.753116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.753428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.754217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:22.754582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.755954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.770492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 255/527] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:22.771244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.780269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a237417-b510-4b5d-bfac-b8e77fddf810 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.818954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.819605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.819888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.820164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.878426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f010ca1-27a6-45b7-b176-d446cf101e07 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:22.889261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.890775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.914819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:22.915584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:22.915866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:22.916141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:22.916969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=aaba1a8ba921489e96f9fa4acecffb74) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:22.917314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:22.919005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:22.933292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 256/528] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/role_assignments?user.id=aaba1a8ba921489e96f9fa4acecffb74&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:22.933292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:22.943634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:22.966363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:23.026497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 273/529] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:23.027647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:23.068368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a782de0-f923-46f0-adcf-6a87451e38b3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.116623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:23.117647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:23.118854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:23.119285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:23.153300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58eb22-3478-44de-92f1-b00fbd3512e9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.196913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:23.197675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:23.197964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:23.198243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:23.199089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=8de75a1df6784111aabec60da40ab756) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:23.199402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:23.200865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:23.207937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 274/530] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:23 2022] GET /identity/v3/role_assignments?user.id=8de75a1df6784111aabec60da40ab756&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:23.208814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:23.215546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:23.217233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:23.229958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:23.247203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:23.341318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5df9-1092-45b1-84e9-020033e82620 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.343175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 257/531] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:23.354611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:23.383446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:23.384352 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:23.384640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:23.384919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:23.397825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:23.399514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:23.419929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:23.420890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:23.421809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:23.421809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:23.421809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:23.421809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:23.425635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 275/532] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:23.427372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:23.523970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:23.535366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a791028c-5cfb-4536-bd24-24ce31e94170 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.553964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:23.574931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:23.576033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:23.576498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:23.576947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:23.587973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f530-9753-46f2-a51e-60d58dc03bf9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.589289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:23.591191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:23.613103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:23.614033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:23.614870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:23.614870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:23.614870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:23.614870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:23.616942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 258/533] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:23.622357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:23.682608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:23.683239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:23.683527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:23.683807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:23.684568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:23.684863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:23.686286 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:23.717610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 276/534] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:23.718559 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:23.732362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7e178-bbc0-46ef-8de5-e38eed78d03c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.769068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:23.770259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:23.770758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:23.771385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:23.772448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:23.772901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:23.774589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:23.788650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 259/535] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:23.794219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:23.825679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:23.893819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3b4b0-413c-46c4-b95f-3b492c752bc2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.932777 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ee022-6af2-440d-b2be-96bdc95923c5 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:23.946825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:23.947625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:23.947907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:23.948182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:23.961716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:23.963177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:23.973596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:23.995961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:23.996844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:23.997740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:24.005047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 277/536] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:24.007250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:24.009438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:24.010239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:24.010522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:24.010798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:24.026323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:24.029484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:24.053076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:24.053973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:24.054842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 260/537] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:24.060054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:24.147663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:24.169034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dfcbc-02bd-40e4-8701-fe3e4eb1da56 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:24.207888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:24.208566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:24.208945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:24.209290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:24.210170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:24.210486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:24.212266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:24.226302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 261/538] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:24.232482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:24.287425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb354f-a213-4a7f-8b3a-2528f4cdbd5a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:24.342832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072f286-840d-4a59-a751-33c23e242da4 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:24.358905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e564541-0993-47b5-8b0d-f0cde47eb1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:24.359588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e564541-0993-47b5-8b0d-f0cde47eb1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:24.359880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e564541-0993-47b5-8b0d-f0cde47eb1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:24.360163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e564541-0993-47b5-8b0d-f0cde47eb1d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:24.360968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e564541-0993-47b5-8b0d-f0cde47eb1d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:24.361293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e564541-0993-47b5-8b0d-f0cde47eb1d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:24.363178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e564541-0993-47b5-8b0d-f0cde47eb1d0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:24.382239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:24.382908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:24.383199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:24.383484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:24.384329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=9937f1088b304574935fcdf579e4de7d) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:24.384647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:24.386218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:24.392565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 262/539] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:24 2022] GET /identity/v3/role_assignments?user.id=9937f1088b304574935fcdf579e4de7d&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:24.397981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 278/540] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:24.403700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:24.429692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:24.607075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd0e3-c233-4009-ac24-5f5de96b445b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:24.653717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f01d8-5ccb-40d5-be7a-17e9c047ec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:24.654384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f01d8-5ccb-40d5-be7a-17e9c047ec93 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:24.654667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f01d8-5ccb-40d5-be7a-17e9c047ec93 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:24.654939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f01d8-5ccb-40d5-be7a-17e9c047ec93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:24.655774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f01d8-5ccb-40d5-be7a-17e9c047ec93 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=9af9eae0fa5b437e96c1c732dfe4fa7a) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:24.656084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f01d8-5ccb-40d5-be7a-17e9c047ec93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:24.665765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f01d8-5ccb-40d5-be7a-17e9c047ec93 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:24.677158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 263/541] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:24 2022] GET /identity/v3/role_assignments?user.id=9af9eae0fa5b437e96c1c732dfe4fa7a&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:25.141496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf891cc2-5ea7-40e1-a8c8-65b49bbbd327 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.141496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf891cc2-5ea7-40e1-a8c8-65b49bbbd327 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.141496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf891cc2-5ea7-40e1-a8c8-65b49bbbd327 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.141496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 279/542] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:25.147741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035342-3007-4b59-9f59-732174ad0d79 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.148171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035342-3007-4b59-9f59-732174ad0d79 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.148353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035342-3007-4b59-9f59-732174ad0d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.170133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-94035342-3007-4b59-9f59-732174ad0d79 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:25.171570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-94035342-3007-4b59-9f59-732174ad0d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:25.260271 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:25.271523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 264/543] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:25.319852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf891cc2-5ea7-40e1-a8c8-65b49bbbd327 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:25.406417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:25.451900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179219-00ba-4cbd-9073-a600d339a5dd None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.452550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179219-00ba-4cbd-9073-a600d339a5dd None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.453169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc179219-00ba-4cbd-9073-a600d339a5dd None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.456250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 265/544] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:25.460704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8d18-4220-44bd-9a09-365dd3b9aa44 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.461224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8d18-4220-44bd-9a09-365dd3b9aa44 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.461750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab8d18-4220-44bd-9a09-365dd3b9aa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.462331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf891cc2-5ea7-40e1-a8c8-65b49bbbd327 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:25.486599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-70ab8d18-4220-44bd-9a09-365dd3b9aa44 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:25.488241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-70ab8d18-4220-44bd-9a09-365dd3b9aa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:25.522079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:25.522868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.523149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.523423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.542852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:25.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:25.552339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:25.553140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 280/545] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:25.563210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:25.609431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 266/546] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:25.688527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8d18-4220-44bd-9a09-365dd3b9aa44 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:25.708299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15d84-e170-48c4-963e-d94d084ae530 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:25.729659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:25.765948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:25.767325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.767729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.768003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.769066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:25.769878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:25.771671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:25.788397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 281/547] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:25.798388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:25.824623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab8d18-4220-44bd-9a09-365dd3b9aa44 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:25.851986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:25.860747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d16cfc8-f10e-43c5-b148-5d1020236ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:25.861578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16cfc8-f10e-43c5-b148-5d1020236ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.861867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16cfc8-f10e-43c5-b148-5d1020236ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.862689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d16cfc8-f10e-43c5-b148-5d1020236ae0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.886381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d16cfc8-f10e-43c5-b148-5d1020236ae0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-9d16cfc8-f10e-43c5-b148-5d1020236ae0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:25.896521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:25.897517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:25.898861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 267/548] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:25.900138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0763a96-3079-4f75-96a3-39b8aa43000d None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:25.900392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0763a96-3079-4f75-96a3-39b8aa43000d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:25.900729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0763a96-3079-4f75-96a3-39b8aa43000d None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:25.905582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 268/549] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:25.913778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0763a96-3079-4f75-96a3-39b8aa43000d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:25.970466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db92a8d-2cc8-4060-b4c5-866c5d5a8c23 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.009147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38875997-f71e-4f9c-b0af-f2118f9c92f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.010000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875997-f71e-4f9c-b0af-f2118f9c92f9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.010285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875997-f71e-4f9c-b0af-f2118f9c92f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.010566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38875997-f71e-4f9c-b0af-f2118f9c92f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.025135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38875997-f71e-4f9c-b0af-f2118f9c92f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.026646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38875997-f71e-4f9c-b0af-f2118f9c92f9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.038449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0763a96-3079-4f75-96a3-39b8aa43000d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-38875997-f71e-4f9c-b0af-f2118f9c92f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:26.045223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:26.047264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 282/550] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc477d8-5291-4b8e-9ef3-a3c4ea0d59ab None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc477d8-5291-4b8e-9ef3-a3c4ea0d59ab None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.049388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc477d8-5291-4b8e-9ef3-a3c4ea0d59ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.066952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fdc477d8-5291-4b8e-9ef3-a3c4ea0d59ab None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:26.068307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fdc477d8-5291-4b8e-9ef3-a3c4ea0d59ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:26.075470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.076247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.076521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.076802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.077582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:26.077895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.079386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.093522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 269/551] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:26.094539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.098850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.149298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 283/552] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:26.149298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc477d8-5291-4b8e-9ef3-a3c4ea0d59ab None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.168991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.255186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ca610-4e15-43da-8f93-9b75b63051c0 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.261676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc477d8-5291-4b8e-9ef3-a3c4ea0d59ab None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.291515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98762942-af71-45eb-877f-defdc9037945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.292189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762942-af71-45eb-877f-defdc9037945 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.292474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762942-af71-45eb-877f-defdc9037945 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.292752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98762942-af71-45eb-877f-defdc9037945 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.293524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98762942-af71-45eb-877f-defdc9037945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:26.293844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98762942-af71-45eb-877f-defdc9037945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.295222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98762942-af71-45eb-877f-defdc9037945 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.308262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.308706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 270/553] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:26.309582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.309712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98762942-af71-45eb-877f-defdc9037945 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.310228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.310663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.327718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.330035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:26.354710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:26.355853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:26.357011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:26.357011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:26.357011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:26.357011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:26.364899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 284/554] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:26.364899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.394952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.429026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98762942-af71-45eb-877f-defdc9037945 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.440088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.465925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.466953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.467330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.467678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.479562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:26.489059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:26.490041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:26.490894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 271/555] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:26.491593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18045e0-be1a-4864-8107-0ded8f06e138 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.491668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.529299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.530141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.530527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.530810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.539127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.540815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:26.557345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:26.559555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 285/556] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:26.562187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.601573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac423ca-b782-4ede-87a2-818bb3458e27 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.640034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.640707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.641118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.641501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.642498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:26.642810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.644209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.646160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.659215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 272/557] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:26.660490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.668894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb58a4-b568-4e31-aae5-6a36efed4507 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.705097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.715137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.716256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.716545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.717016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.717827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:26.718505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.720468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.733242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 286/558] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:26.734394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.781346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643016a0-97e5-4366-b141-04c2aabfb17a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.822633 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.823554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.823958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.824330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.825296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:26.825768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.827387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.847368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 273/559] 10.222.0.35 () {56 vars in 1135 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:26.848362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.875256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723bcb23-9b3d-42f3-8412-8b3f516ae85d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:26.917074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:26.918052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:26.918465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:26.918745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:26.928364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:26.930077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:26.932638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:26.948828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:26.951732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 287/560] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:26.952685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:26.975954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:26.995590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27068e-f39f-4c4f-8e5d-3120d0860fa2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.048781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.051549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.052040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.052739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.068107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.069502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a75009-255f-4ebf-9eaa-2704a1f73415 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.070160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:27.095555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:27.097213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:27.098502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:27.098502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:27.098502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:27.098502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:27.099303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 274/561] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:27.099727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.117994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e75f-73eb-4f40-8532-d9f2a74b225f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.118635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e75f-73eb-4f40-8532-d9f2a74b225f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.118945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e75f-73eb-4f40-8532-d9f2a74b225f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.119221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e75f-73eb-4f40-8532-d9f2a74b225f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.193210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:27.197909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0e75f-73eb-4f40-8532-d9f2a74b225f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.199432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d0e75f-73eb-4f40-8532-d9f2a74b225f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.223056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d702197-555d-4901-a23c-9ef9c67e95df None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.314376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.314963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.315959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.316229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.317031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:27.317365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.318788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.328655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:27.341750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 275/562] 10.222.0.35 () {56 vars in 1137 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:27.341750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.350863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 288/563] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:27.352865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95297-e080-4f89-be69-be45a41c29a8 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.353151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95297-e080-4f89-be69-be45a41c29a8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.353450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a95297-e080-4f89-be69-be45a41c29a8 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.357391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 289/564] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:27.358419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95297-e080-4f89-be69-be45a41c29a8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.447303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e6220-3797-4e3f-b0cd-4beef152ada3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.457016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:27.461512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a95297-e080-4f89-be69-be45a41c29a8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.480468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.481124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.481433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.481703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.482423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:27.482727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.484086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.496128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 276/565] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:27.496476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f42d9-4b55-4820-a800-3aaddc2cd3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.497128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.497249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f42d9-4b55-4820-a800-3aaddc2cd3fa None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.497579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f42d9-4b55-4820-a800-3aaddc2cd3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.497909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f42d9-4b55-4820-a800-3aaddc2cd3fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.508364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f42d9-4b55-4820-a800-3aaddc2cd3fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.509865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f42d9-4b55-4820-a800-3aaddc2cd3fa None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-f54f42d9-4b55-4820-a800-3aaddc2cd3fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:27.528732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:27.529693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:27.530607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:27.530607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:27.530607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:27.530607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:27.530607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 290/566] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:27.531142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f31ea2-461c-4a1d-ac12-08ab5fcdccfc None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.531425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f31ea2-461c-4a1d-ac12-08ab5fcdccfc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.531698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f31ea2-461c-4a1d-ac12-08ab5fcdccfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.548208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e3f31ea2-461c-4a1d-ac12-08ab5fcdccfc None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:27.549478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e3f31ea2-461c-4a1d-ac12-08ab5fcdccfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:27.559107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:27.601867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382a91f7-14f6-483f-a80e-958d00cc09a9 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.616266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 291/567] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:27.617611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0d70-8376-4e69-994a-87205b079aba None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.617901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0d70-8376-4e69-994a-87205b079aba None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.618209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a0d70-8376-4e69-994a-87205b079aba None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.619474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 292/568] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:27.620169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0d70-8376-4e69-994a-87205b079aba None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.637786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.638610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.638890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.639161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.650144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.651581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:27.668895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:27.669982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:27.670880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:27.670880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:27.670880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:27.670880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:27.670880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 277/569] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:27.670880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.713431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:27.732773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a0d70-8376-4e69-994a-87205b079aba None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.767642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11562a-5905-4a66-ae40-4b02c72b018e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.769041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11562a-5905-4a66-ae40-4b02c72b018e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.769358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11562a-5905-4a66-ae40-4b02c72b018e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.769636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11562a-5905-4a66-ae40-4b02c72b018e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.770032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59c49b-911f-486d-913e-30b3165a7191 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.776746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11562a-5905-4a66-ae40-4b02c72b018e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.778226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11562a-5905-4a66-ae40-4b02c72b018e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.780062 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-ad11562a-5905-4a66-ae40-4b02c72b018e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:27.792743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:27.793745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:27.794713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 293/570] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:27.795307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdbfc1-fa3f-4ca7-a66b-c9f0e9f93b32 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.795383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdbfc1-fa3f-4ca7-a66b-c9f0e9f93b32 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.795787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdbfc1-fa3f-4ca7-a66b-c9f0e9f93b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.807340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.808007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.808292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.808571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.809326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:27.809638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:27.810990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:27.812223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-56fdbfc1-fa3f-4ca7-a66b-c9f0e9f93b32 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:27.813430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-56fdbfc1-fa3f-4ca7-a66b-c9f0e9f93b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:27.825089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 278/571] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:27.825846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.883665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 294/572] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:27.884418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdbfc1-fa3f-4ca7-a66b-c9f0e9f93b32 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:27.925425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ded3a3-f6e5-4a35-b840-cabc2f6ab804 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.953074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:27.959529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:27.960409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:27.960593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:27.982489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:27.983702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:27.984655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:27.985646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 279/573] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:27.985715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdbfc1-fa3f-4ca7-a66b-c9f0e9f93b32 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:27.985802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.013248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.020997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.021691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.021972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.022541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.022962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:28.023259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.024629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.036153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 295/574] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:28.036949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.087151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92810a71-89f5-4ed1-9b39-647d57e610f5 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.120841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.121540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.121826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.122106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.122945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:28.123255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.124807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.133044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd3bf1-fff2-40be-b510-9a14a34304a2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.147764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 280/575] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:53:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:28.147764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.161990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.167742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.168534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.168814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.169085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.176020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.182175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:28.192471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:28.194004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 296/576] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:28.194907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.237435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.248803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de4cc51-bb83-4f4c-acd2-a814a7a31dcb None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.285589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.286385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.286677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.286958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.294886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d3d22-d921-4770-90e9-677d0b606113 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.296948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:28.305044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:28.306054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:28.306897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 281/577] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:28.307590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.333257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.333972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.334255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.334534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.335291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:28.335596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.336982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.345150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 297/578] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:28.345757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.400369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.412165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4446603-b9a2-4a10-87be-3885c1afbcd2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.445682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764da2ef-19ad-46a6-bf44-75a3742dbd23 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.446296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.447093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.447377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.447658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.454190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.455629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:28.470176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:28.471127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 282/579] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:28.472386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.482530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.484211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.484887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.485175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.485459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.496389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.497897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:28.515273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:28.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:28.517935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:28.517935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:28.517935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:28.517935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:28.517935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 298/580] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:28.517935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.575022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fa6c3-ac2a-449c-a3b1-2bdfc894ac80 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.609641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.610295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.610579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.610851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.611630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:28.611939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.613680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.618595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d34d6-fb1b-45cc-88a7-54533b8aae16 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.620868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.624987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 283/581] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:28.631590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.658677 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.659839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.660125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.660405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.661159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:28.661495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.662868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.670981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 299/582] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:28.671774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.692085 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.725236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4cf129-6f28-44c8-bb4b-eba2889f264b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.758352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.759138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.759424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.759708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.770890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.771958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a15b88-48fe-4c54-910d-4db19c9aea88 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.772333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:28.789611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:28.790681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:28.791623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:28.791623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:28.791623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:28.791623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:28.791623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 284/583] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:28.792103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.806141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.806804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.807087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.807365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.808152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:28.808461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.809933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.822147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 300/584] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:28.822908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.850844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.889824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b77f8d-89a7-4eef-899b-4ce5523fc5bd None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.910639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:28.922516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fc2e2a-8546-4c2d-801b-848743546c08 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:28.924312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.925279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.925586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.925861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.926578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:28.926883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.928246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:28.940840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 285/585] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:28.941670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:28.955616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:28.957163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:28.957466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:28.957735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/aaba1a8ba921489e96f9fa4acecffb74/roles/36d82dc24b2b44ee935e79f13d0f521c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:28.972339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=aaba1a8ba921489e96f9fa4acecffb74, role_id=36d82dc24b2b44ee935e79f13d0f521c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:28.973840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.039536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f258d8-52f4-4cb0-ad14-33eaeb101e10 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.044055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 301/586] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:28 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/aaba1a8ba921489e96f9fa4acecffb74/roles/36d82dc24b2b44ee935e79f13d0f521c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:29.044863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.077089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.078099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.078535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.078823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.091626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.095378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:29.114964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:29.115960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:29.116847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:29.116847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:29.116847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:29.116847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:29.116847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 286/587] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:29.117131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.139669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:29.158047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:29.172165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774f3d02-3f3e-4f00-aca3-15648ee335e7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.214286 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.215166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.215249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.215492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.216239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:29.216547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.217989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.232027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 302/588] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:29.233419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.253845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f3a09-1f6d-489f-b7fa-e5e2fcd714fe None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.291791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.292435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.292717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.292997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.293858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=05f46d019e1348adabd023de481f0a3d) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:29.294172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.295698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.309308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 287/589] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/role_assignments?user.id=05f46d019e1348adabd023de481f0a3d&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:29.309308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.371503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da712b-6728-45aa-9ec3-79b741848df9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.411101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:29.417332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727d5f24-59e8-437a-9837-5021f5cbe22a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.423097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.423908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.424193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.425201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.430580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:29.437398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.440113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.460120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.460792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.461142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.461454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:29.462078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:29.462952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:29.464234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:29.464686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:29.464686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.465171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 303/590] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:29.465341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.465555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.482336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 288/591] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:29.483287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.592581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959c1cee-96af-454c-a053-cab08db46640 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.628660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.629660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.630043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.630418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.631506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f666367-8d5a-408c-bb47-a13d0e80efd6 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.638389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.639961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:29.656049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:29.657073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:29.658334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 304/592] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:29.663171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.670725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.671328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.671614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.671894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.673202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:29.673555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.676189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.697444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 289/593] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:29.698450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.723370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:29.754275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:29.809554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5479dfb-a08e-41b6-bd9f-76ff75d86669 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.824219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aaaf13-fd44-4b3c-a8b3-624cc7391533 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.856355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.857566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.858026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.858457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.860707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:29.861325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:29.861614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:29.861899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:29.862817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:29.863217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.864653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.867832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:29.869513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:29.885455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:29.886720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 09:53:29.887869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:29.888499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 290/594] 10.222.0.35 () {56 vars in 1139 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:29.889038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.890594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 305/595] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:29.891533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:29.971459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:29.996941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21564acc-e9bb-4e56-b896-5e7bb91418e1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:29.999814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b699c1d6-7c67-4883-8112-861a28fb7bb4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.018263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:30.034529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.035477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.035763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.036040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/8de75a1df6784111aabec60da40ab756/roles/36d82dc24b2b44ee935e79f13d0f521c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.038027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.038986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.039439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.039858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.051737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.053649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.059659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=8de75a1df6784111aabec60da40ab756, role_id=36d82dc24b2b44ee935e79f13d0f521c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.061118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:30.072937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:30.078072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:30.079355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:30.079355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:30.079355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:30.079355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:30.079355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 306/596] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:30.079355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:30.137444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 291/597] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:29 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/8de75a1df6784111aabec60da40ab756/roles/36d82dc24b2b44ee935e79f13d0f521c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:30.138816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:30.282963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3975db51-936f-45a2-9068-3fc6546e50f5 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.286579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b53e9-ad16-4cd5-a179-abc7d9fa3302 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.313704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:30.325024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.326125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.333728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.333728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.333728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:30.333728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.333728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.344620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 292/598] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:30.350944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:30.395790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.396517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.396797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.397071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.398036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:30.398348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.399841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.436769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 307/599] 10.222.0.35 () {56 vars in 1137 bytes} [Tue May 24 09:53:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:30.443056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:30.471070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e033801b-d911-4b60-87fb-1405ce1bb977 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.478211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:30.520337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.521282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.521825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.522110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.532250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.533829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:30.556347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:30.557344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:30.558252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:30.559181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 293/600] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:30.563966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:30.588486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec2b64-29fe-4395-9317-4c32a6e661d0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.646686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:30.661650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.662458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.662739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.666834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.686928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.688456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:30.725785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:30.726796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:30.728140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:30.728140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:30.728140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:30.728140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:30.728140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 308/601] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:30.737515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110fc52b-59fe-4cbd-87a5-2390cafc381b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.741205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:30.774120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.774842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.775132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.775411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.776212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:30.776521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.778051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.791074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 294/602] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:30.796761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:30.876838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:30.907156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20fb6d-e3b8-4287-9c4b-76cc7ecb4c72 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.975095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:30.975775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:30.976060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:30.976335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:30.977067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:30.977399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:30.978789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:30.985412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7219e581-abcc-4d8b-8cfc-fe7798f3739c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:30.988574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:31.001124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 309/603] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:31.007355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:31.024290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:31.025577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.026184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.026675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/9937f1088b304574935fcdf579e4de7d/roles/36d82dc24b2b44ee935e79f13d0f521c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.052808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=9937f1088b304574935fcdf579e4de7d, role_id=36d82dc24b2b44ee935e79f13d0f521c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:31.057839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:31.159241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 295/604] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:30 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/9937f1088b304574935fcdf579e4de7d/roles/36d82dc24b2b44ee935e79f13d0f521c => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:31.161341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b9277-1143-4185-8a8a-9cfedbecf0fb None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:31.209038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0cd26a-efb7-42e8-9087-34ccc12708ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:31.209895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cd26a-efb7-42e8-9087-34ccc12708ff None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.210188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cd26a-efb7-42e8-9087-34ccc12708ff None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.210468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0cd26a-efb7-42e8-9087-34ccc12708ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.218468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cd26a-efb7-42e8-9087-34ccc12708ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:31.219945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0cd26a-efb7-42e8-9087-34ccc12708ff None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:31.223754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-4e0cd26a-efb7-42e8-9087-34ccc12708ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:31.241623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:31.242596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:31.243493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:31.245601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 310/605] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:31.248689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:31.410393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2eb23b-be92-452a-b4a0-ec82c0292dbf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:31.420828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd1a9e-9e3c-43fa-94c0-7762cd42b2ed None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.421419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd1a9e-9e3c-43fa-94c0-7762cd42b2ed None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.421419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd1a9e-9e3c-43fa-94c0-7762cd42b2ed None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.423055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 311/606] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:31.428510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9609012-7d82-4150-a11a-a3bed7752f96 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.428784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9609012-7d82-4150-a11a-a3bed7752f96 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.429056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9609012-7d82-4150-a11a-a3bed7752f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.457869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c9609012-7d82-4150-a11a-a3bed7752f96 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:31.459066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c9609012-7d82-4150-a11a-a3bed7752f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:31.462624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:31.464564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.465075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.465577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.466580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:31.467097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:31.469339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:31.487535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 296/607] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:31.494584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:31.522138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:31.547053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 312/608] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:31.570765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9609012-7d82-4150-a11a-a3bed7752f96 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:31.625558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d1648-25f5-4089-b9e4-d4047788a4e1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:31.665714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a576ba-b2d6-41a2-a256-bcd0bb186df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:31.667287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a576ba-b2d6-41a2-a256-bcd0bb186df3 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.667783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a576ba-b2d6-41a2-a256-bcd0bb186df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.668282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a576ba-b2d6-41a2-a256-bcd0bb186df3 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/9af9eae0fa5b437e96c1c732dfe4fa7a/roles/36d82dc24b2b44ee935e79f13d0f521c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.687617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a576ba-b2d6-41a2-a256-bcd0bb186df3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=9af9eae0fa5b437e96c1c732dfe4fa7a, role_id=36d82dc24b2b44ee935e79f13d0f521c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:31.689426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a576ba-b2d6-41a2-a256-bcd0bb186df3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:31.759837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 297/609] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:31 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/9af9eae0fa5b437e96c1c732dfe4fa7a/roles/36d82dc24b2b44ee935e79f13d0f521c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:53:31.760752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9609012-7d82-4150-a11a-a3bed7752f96 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:31.783854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db69dc-554d-49d2-8362-06ee76c983be None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.784176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db69dc-554d-49d2-8362-06ee76c983be None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.784490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96db69dc-554d-49d2-8362-06ee76c983be None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.786211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 298/610] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:31.791784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86742-09ca-410d-8959-3ca9f9005331 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.792075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86742-09ca-410d-8959-3ca9f9005331 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.794129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86742-09ca-410d-8959-3ca9f9005331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.795212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:31.810696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:31.812060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:31.812172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0fa86742-09ca-410d-8959-3ca9f9005331 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:31.812656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:31.813187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:31.813408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0fa86742-09ca-410d-8959-3ca9f9005331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:31.839355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:31.855499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:31.856533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:31.857411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:31.861512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 313/611] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:31.868381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:31.945680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:31.971116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 299/612] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:32.003315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72c3aa-662a-4e33-9cc5-d64c108f596f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:32.006845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa86742-09ca-410d-8959-3ca9f9005331 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:32.043876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:32.044910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:32.045526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:32.046011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:32.046979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:32.047468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:32.049036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:32.059475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 314/613] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:32.064837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:32.173527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:32.199131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa86742-09ca-410d-8959-3ca9f9005331 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:32.246924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:32.247743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:32.248093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:32.248298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:32.252049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db77f-db3e-42b6-be69-1cc32066c0a4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:32.277938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:32.286836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:32.287800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:32.288635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 300/614] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:32.292063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:32.293036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:32.293500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:32.293914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:32.296158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:32.307334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:32.309029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:32.327777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:32.328945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:32.330065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:32.330065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:32.330065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:32.330065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:32.333034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 315/615] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:32.339775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:32.403889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:32.460783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b078b34-e761-4530-9fe0-0bdae12a9e63 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:32.500093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:32.501022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:32.501326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:32.501609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:32.503480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:32.503797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:32.505433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:32.515393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 301/616] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:32.520729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:32.540397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f424ed4-5db9-4e93-845d-14ee74832b0a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:32.542735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:32.617970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:32.618653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:32.618934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:32.619209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:32.619947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:32.620255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:32.621657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:32.663708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec6362-874c-4a63-927c-975450fe5bbf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:32.667559 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 316/617] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:32.673942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:32.708552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:32.709397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:32.709688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:32.709972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:32.721806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:32.726002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:32.727464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:32.755405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:32.756388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:32.757411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:32.757411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:32.757411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:32.757411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:32.758141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 302/618] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:32.763827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:32.970056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc93fd-384c-4e92-a5c9-83095e691884 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.018220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f42d1e-3e89-4f30-9211-b95fff0b1e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.020829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42d1e-3e89-4f30-9211-b95fff0b1e58 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.020829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42d1e-3e89-4f30-9211-b95fff0b1e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.020829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f42d1e-3e89-4f30-9211-b95fff0b1e58 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.028715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f42d1e-3e89-4f30-9211-b95fff0b1e58 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:33.030530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f42d1e-3e89-4f30-9211-b95fff0b1e58 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:33.048347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7f6ad-4ca1-468c-8d8c-89956ef7deb9 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-73f42d1e-3e89-4f30-9211-b95fff0b1e58 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:33.062137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:33.063132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:53:33.064097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:33.077289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 317/619] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:33.078806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cc528-8646-489b-baa8-8e0af13c83f4 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.080285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cc528-8646-489b-baa8-8e0af13c83f4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.081299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cc528-8646-489b-baa8-8e0af13c83f4 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.086961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 318/620] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:33 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:33.087696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cc528-8646-489b-baa8-8e0af13c83f4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:33.110648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6535f-e7c9-427d-83d7-2f738eb44378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.111330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6535f-e7c9-427d-83d7-2f738eb44378 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.117287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6535f-e7c9-427d-83d7-2f738eb44378 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.117287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6535f-e7c9-427d-83d7-2f738eb44378 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.117287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6535f-e7c9-427d-83d7-2f738eb44378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:33.118073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6535f-e7c9-427d-83d7-2f738eb44378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:33.119336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:33.120131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6535f-e7c9-427d-83d7-2f738eb44378 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:33.171809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 303/621] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:33.173176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ade66-699e-4f6f-a276-1e383b2ba5d4 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.173848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ade66-699e-4f6f-a276-1e383b2ba5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.174131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ade66-699e-4f6f-a276-1e383b2ba5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.199384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9b9ade66-699e-4f6f-a276-1e383b2ba5d4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:33.200713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9b9ade66-699e-4f6f-a276-1e383b2ba5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:33.220049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cc528-8646-489b-baa8-8e0af13c83f4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.240487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:33.268014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.269576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.270020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.270435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.271992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:33.272537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:33.274480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:33.303308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 319/622] 10.222.0.35 () {56 vars in 1137 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:33.304929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:33.329321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 304/623] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:33.329321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ade66-699e-4f6f-a276-1e383b2ba5d4 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:33.428347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:33.449895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714f5b3-1975-4fc6-b29e-51139b52d57e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.453890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ade66-699e-4f6f-a276-1e383b2ba5d4 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.495219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.496936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.497441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.497709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.498181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.498297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.498504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.498778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.506915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:33.508504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:33.524274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:33.525301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:33.526287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 320/624] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:33.526902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:33.537985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:33.594903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:33.598505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:33.649334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338bb8da-5af2-499c-ae76-cf91c03b62cf None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.703909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.705520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.706062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.706552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.712228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 305/625] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:33.713284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:33.718298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:33.723213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:33.735011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:33.736045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:33.736919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:33.745449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 321/626] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:33.745449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:33.785073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:33.831721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc1c48-1500-4c50-b7b9-dc37bf0eb83a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.868762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.869293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.869598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.869875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.870911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:33.871175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:33.872800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:33.885698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f2095-8ea3-49eb-9f2d-64df8186bc31 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:33.891754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 306/627] 10.222.0.35 () {56 vars in 1135 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:33.892852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:33.925801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c025e2c5-742d-4c24-a3b1-4cc6e4f64d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:33.926764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e2c5-742d-4c24-a3b1-4cc6e4f64d2e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.927122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e2c5-742d-4c24-a3b1-4cc6e4f64d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.927402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c025e2c5-742d-4c24-a3b1-4cc6e4f64d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.938601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025e2c5-742d-4c24-a3b1-4cc6e4f64d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:33.940080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c025e2c5-742d-4c24-a3b1-4cc6e4f64d2e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c025e2c5-742d-4c24-a3b1-4cc6e4f64d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:33.958140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:33.959194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:33.960115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:33.960115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:33.960115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:33.960115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:33.960115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 322/628] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:33.960810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0c8e-19ca-4871-81ad-376a09f124ef None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:33.960810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0c8e-19ca-4871-81ad-376a09f124ef None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:33.961191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b0c8e-19ca-4871-81ad-376a09f124ef None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:33.965389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 323/629] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:33.965389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b0c8e-19ca-4871-81ad-376a09f124ef None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.006804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47844a-ce2e-4ecb-837d-f1d31a0d75d6 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.023284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.025909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.048284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.048284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.048284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.048284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.048683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:34.048683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.048683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.061231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 307/630] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:34.061231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.075166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520b0c8e-19ca-4871-81ad-376a09f124ef None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.112186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de7d4e-2fee-4c6c-80c5-1f5e1b5c00a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.112986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de7d4e-2fee-4c6c-80c5-1f5e1b5c00a0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.113295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de7d4e-2fee-4c6c-80c5-1f5e1b5c00a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.113587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de7d4e-2fee-4c6c-80c5-1f5e1b5c00a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.126288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de7d4e-2fee-4c6c-80c5-1f5e1b5c00a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.127805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8de7d4e-2fee-4c6c-80c5-1f5e1b5c00a0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-f8de7d4e-2fee-4c6c-80c5-1f5e1b5c00a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:34.145974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:34.146992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:34.148064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:34.148064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:34.148064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:34.148064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:34.149594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 324/631] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:34.150958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf8e3e-2692-4035-b103-cfe2f19cefd3 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.151239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf8e3e-2692-4035-b103-cfe2f19cefd3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.151513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf8e3e-2692-4035-b103-cfe2f19cefd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.169327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5bbf8e3e-2692-4035-b103-cfe2f19cefd3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:34.170673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5bbf8e3e-2692-4035-b103-cfe2f19cefd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:34.178775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231beb5-e04a-4838-ac11-03b8d44bbc67 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.219658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.220341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.220829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.221048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.221907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:34.222216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.223655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.241871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 308/632] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:34.241871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.254540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 325/633] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:34.261484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf8e3e-2692-4035-b103-cfe2f19cefd3 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.289670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.305009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.364401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c47d1-c6de-449e-9b21-f93fcac5088b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.377059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf8e3e-2692-4035-b103-cfe2f19cefd3 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.401256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.402081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.402371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.402658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.413013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.413499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.413704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.413988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.414264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.414978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.415076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:34.415288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.416676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.428372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 326/634] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:34.429165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:34.433141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:34.434204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:34.435165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:34.435165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:34.435165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:34.435165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:34.435165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 309/635] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:34.435165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.524768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.527345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.533459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f512f9-6f41-46f6-a456-2ac30f68113d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.534976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091a787-ec6a-48ba-bf74-9cf7906290a9 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.567202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.568044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.568328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.568603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.569520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb605a-5b6f-49a6-bdf5-a94b858e9364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.570404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb605a-5b6f-49a6-bdf5-a94b858e9364 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.570697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb605a-5b6f-49a6-bdf5-a94b858e9364 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.570978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb605a-5b6f-49a6-bdf5-a94b858e9364 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.575527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.577591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.581873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bb605a-5b6f-49a6-bdf5-a94b858e9364 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-f2bb605a-5b6f-49a6-bdf5-a94b858e9364 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:34.589503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:34.590641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:34.591492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 310/636] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:34.592533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2631d-f25e-481f-ae89-afd79663ec94 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:34.592613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:34.599776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:34.600974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:34.601562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2631d-f25e-481f-ae89-afd79663ec94 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.601562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae2631d-f25e-481f-ae89-afd79663ec94 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.601716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 327/637] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:34.601716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.601882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 311/638] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:34.601882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2631d-f25e-481f-ae89-afd79663ec94 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.697833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f88b6e-1c6c-4fdc-8768-e391751b30fc None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.700752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae2631d-f25e-481f-ae89-afd79663ec94 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.731585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b318b-65ca-42c6-84ec-a9ef79c26b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.732410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b318b-65ca-42c6-84ec-a9ef79c26b3c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.732690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b318b-65ca-42c6-84ec-a9ef79c26b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.733026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b318b-65ca-42c6-84ec-a9ef79c26b3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.734114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.734767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.735047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.735319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.736035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:34.736367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.738017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.739636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b318b-65ca-42c6-84ec-a9ef79c26b3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.741063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b318b-65ca-42c6-84ec-a9ef79c26b3c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.749545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 312/639] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:34.750325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-600b318b-65ca-42c6-84ec-a9ef79c26b3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:34.755342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:34.756324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:34.757233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 328/640] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:34.758074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18774c-783f-4ead-878d-e6922da067c2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.758074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18774c-783f-4ead-878d-e6922da067c2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.758287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18774c-783f-4ead-878d-e6922da067c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.762444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.775357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fa18774c-783f-4ead-878d-e6922da067c2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:34.776554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fa18774c-783f-4ead-878d-e6922da067c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:34.786508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.843048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 329/641] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:34.844159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18774c-783f-4ead-878d-e6922da067c2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.857419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efd492-8ae4-41e1-b07f-a8f6e9547906 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.892939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.894273 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.894556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.894834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.895581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:34.895888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.897254 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:34.905169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 313/642] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:34.905975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:34.954915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18774c-783f-4ead-878d-e6922da067c2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:34.976219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:34.989133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:34.990884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:34.990884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:34.990884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:34.991223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:34.991427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:34.992871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.004961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 330/643] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:35.005741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab75dba-3f5c-423a-94aa-22e9cdebcef4 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.005845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.010311 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:35.040435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.041227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.041578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.042912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.049573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.051074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:35.064915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:35.066906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 314/644] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:53:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:35.068043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.112072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7dc49-3c97-4930-910c-ec8b92399fa9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.147604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.148263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.148538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.148839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.149662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:35.149969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.151422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.164134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 331/645] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:35.164903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.176471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6249ad75-58d2-4af2-a351-ca805ef4db76 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.205624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:35.211351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.212145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.212420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.212688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.223076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:35.226471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:35.230667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:35.231784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 315/646] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:35.232705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.266927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba58b4-18ec-408c-86ad-1662a71ab66b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.300800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.301617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.301898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.302171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.311941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.313422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.332378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ff62ce-4c77-4d5d-9d92-85c4b7321c8d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:35.332550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:35.333589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:35.334488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:35.334488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:35.334488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:35.334488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:35.334488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 332/647] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:35.334789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.368977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.369585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.369868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.370164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.371013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=aaba1a8ba921489e96f9fa4acecffb74) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:35.371324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.372772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.379613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 316/648] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/role_assignments?user.id=aaba1a8ba921489e96f9fa4acecffb74&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:35.380385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.444539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:35.459166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f279111-8549-487b-8aea-ec451b01cca2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.479608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:35.497876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93e0a8-c5b4-403a-a6a6-b1f82e137240 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.500779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.501568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.501851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.502127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.503101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:35.503435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.505030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.526089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 333/649] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:35.527392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.545513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.546408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.546698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.546958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/05f46d019e1348adabd023de481f0a3d/roles/36d82dc24b2b44ee935e79f13d0f521c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.564122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=05f46d019e1348adabd023de481f0a3d, role_id=36d82dc24b2b44ee935e79f13d0f521c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.564990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.632598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8179b39-eb77-40fd-a73f-9266e7a3deb6 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.666051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.666763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.667048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.667327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.668083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:35.668392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.669801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.674531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 317/650] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:35 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/05f46d019e1348adabd023de481f0a3d/roles/36d82dc24b2b44ee935e79f13d0f521c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:35.677368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.679498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 334/651] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:35.679498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.719410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:35.798288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f083c7b6-0c8f-412e-aa8e-4ba651d08bf2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.821421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a2309-e0bf-49b1-9636-9ca1cd646cf9 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:35.844816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.845528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.845813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.846090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.853683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:35.866023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:35.866751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:35.867032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:35.867305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:35.868048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:35.868356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.869812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.887898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 318/652] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:35.888903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:35.937845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:35.939311 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:35.985031 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.006917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f61d3-312b-43c9-a3c2-c4512324f4cc None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.043472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.044583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.045078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.045651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.057058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.058666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.070688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 335/653] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:36.072028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:36.078559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:36.079452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:36.080354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:36.080354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:36.080354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:36.080354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:36.080354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 319/654] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:36.085858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.106240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.202992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d339962a-af35-40df-b8d2-35ba6bd11c72 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.212949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e70177b-67cc-42c7-8cab-a948e01f9a5f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.245042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.245930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.246210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.246481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.257085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.258242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.258524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.258629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.258803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.260084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.261659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.269162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.271516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:36.282216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:36.283317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:36.284434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:36.284434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:36.284434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:36.284434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:36.288308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 320/655] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:36.289155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:36.290573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:36.291594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 24 09:53:36.292552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:36.296327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 336/656] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:36.297378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.383411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.424320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f2a55-98b8-4352-b6eb-7ef8276a0d68 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.428485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96762e-1816-4b73-80f6-02f220d6551d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.460814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.461776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.462227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.462711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.463685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:36.464135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.465791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.472641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.473545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.473940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.474310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.475781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:36.476189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.478211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.480597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 337/657] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:36.481799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.502316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 321/658] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:36.502316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.553636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.610791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2e7eb-a1b8-470b-8399-053e8c7cef34 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.634564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3d025-f066-4408-a274-c413aa831d5b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.652298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.653746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.654213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.654666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.655679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:36.656691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.658653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.666589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.673300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.674066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.674478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.674957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.682388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.684020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.684668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 338/659] 10.222.0.35 () {56 vars in 1139 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:36.685823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:36.702784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:36.703741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:36.704641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:36.709212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 322/660] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:36.710394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.820736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3e01a-1195-4973-8e2b-c473a29de5bc None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.823368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d18b50-cbc9-4b01-9207-0f2b9dd1a969 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:36.840729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.859977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.860808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.861183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.861618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.865072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:36.866175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:36.874588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:36.874588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:36.875991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.877797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:36.895836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:36.896954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 24 09:53:36.897933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:36.904787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 339/661] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:36.905913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:36.928054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:36.939146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:36.941174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.015911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec842c3-96e0-4e62-93e9-2f6cd4700103 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:37.050999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 323/662] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:37.052046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:37.055566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:37.056489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.056975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.057527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.058659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:37.059171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:37.060959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.080513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 340/663] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:37.082353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:37.119630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:37.149067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:37.159721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a279bea0-70f8-426b-a4fc-26ef64e1e402 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:37.199234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:37.199955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.200234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.200552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.201564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:37.201875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:37.203424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.206455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ace84-5341-4f38-843a-bce462b84804 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:37.223149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 324/664] 10.222.0.35 () {56 vars in 1137 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:37.228268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:37.250376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:37.251490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.251772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.252043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.266971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:37.269030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:37.291464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:37.292450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:37.293838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:37.293838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:37.293838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:37.293838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:37.293838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 341/665] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:37.294153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:37.351155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df9736-a23c-4ec1-808d-4b3d41a4fe00 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:37.389328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:37.390179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.390461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.390780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.391424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:37.393188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=8de75a1df6784111aabec60da40ab756) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:37.394357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:37.395999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.402900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 325/666] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/role_assignments?user.id=8de75a1df6784111aabec60da40ab756&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:37.406798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:37.425237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696055fe-aa45-4aaa-9af8-5bb16d553ad1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:37.435385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:37.484556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f3f50e-770e-4587-84d8-3971b2d16060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:37.485165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f50e-770e-4587-84d8-3971b2d16060 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.485480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f50e-770e-4587-84d8-3971b2d16060 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.485757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3f50e-770e-4587-84d8-3971b2d16060 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.544838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e3a61-96df-4e97-804d-5c4e72f091b2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:37.566704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3f50e-770e-4587-84d8-3971b2d16060 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:37.568201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f3f50e-770e-4587-84d8-3971b2d16060 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.587816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:37.588903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.589424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.589913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.590884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:37.591414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:37.593031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.608572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 326/667] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:37.618305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:37.762157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:37.794099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:37.798132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494fbf7-5dae-49b7-acf7-33795ab0ae91 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:37.807492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 342/668] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:37.808673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c181dcd-69ab-4777-826a-db71a96b29b9 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.808960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c181dcd-69ab-4777-826a-db71a96b29b9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.809254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c181dcd-69ab-4777-826a-db71a96b29b9 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.814337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 343/669] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:37.819662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda740a-b87c-4fac-9d74-753d7062b4ff None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.819950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda740a-b87c-4fac-9d74-753d7062b4ff None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.820315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda740a-b87c-4fac-9d74-753d7062b4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.839100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:37.839779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6cda740a-b87c-4fac-9d74-753d7062b4ff None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:37.840231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:37.840231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:37.840462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:37.840965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6cda740a-b87c-4fac-9d74-753d7062b4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:37.848519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:37.850411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:37.867418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:37.868542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 327/670] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:37.869748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:37.933378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 344/671] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:37.933378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda740a-b87c-4fac-9d74-753d7062b4ff None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.014552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90969a34-f987-483d-8e9b-a02f30b65878 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.047722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.050889 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda740a-b87c-4fac-9d74-753d7062b4ff None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.054030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.055054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.060135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.060135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.068918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.070131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.071869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.090228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb688bc2-444b-4fbb-b296-51c56a01108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.091003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb688bc2-444b-4fbb-b296-51c56a01108c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.091360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb688bc2-444b-4fbb-b296-51c56a01108c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.091710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb688bc2-444b-4fbb-b296-51c56a01108c None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:38.092426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:38.093412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:38.094319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:38.094319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:38.094319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:38.094319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:38.094544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb688bc2-444b-4fbb-b296-51c56a01108c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:38.094544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb688bc2-444b-4fbb-b296-51c56a01108c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.094544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb688bc2-444b-4fbb-b296-51c56a01108c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.097537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 328/672] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:38.098549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.112515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 345/673] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:38.113889 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45638045-6e9f-48c7-8877-449e448adee9 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.114215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45638045-6e9f-48c7-8877-449e448adee9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.114451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45638045-6e9f-48c7-8877-449e448adee9 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.115828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 346/674] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:38.120384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45638045-6e9f-48c7-8877-449e448adee9 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.215846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f14d07d-a0b4-462e-9790-0b000fda2203 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.229385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45638045-6e9f-48c7-8877-449e448adee9 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.254634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.255933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.256363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.256792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.257477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.259078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.263784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a90bd-69b2-4be7-8011-3b7b90043325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.264453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a90bd-69b2-4be7-8011-3b7b90043325 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.264792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a90bd-69b2-4be7-8011-3b7b90043325 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.265011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a90bd-69b2-4be7-8011-3b7b90043325 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.265779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a90bd-69b2-4be7-8011-3b7b90043325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:38.266087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a90bd-69b2-4be7-8011-3b7b90043325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.267705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a90bd-69b2-4be7-8011-3b7b90043325 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:38.276405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:38.277608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:38.278469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:38.280495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 329/675] 10.222.0.35 () {56 vars in 1129 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:38.281433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.287427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 347/676] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:38.288800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c4938-ce2f-4a6b-b8c7-028d80b84ef7 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.289080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c4938-ce2f-4a6b-b8c7-028d80b84ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.289449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c4938-ce2f-4a6b-b8c7-028d80b84ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.306245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8d4c4938-ce2f-4a6b-b8c7-028d80b84ef7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:38.307425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8d4c4938-ce2f-4a6b-b8c7-028d80b84ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:38.335642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.337025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.381357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 348/677] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:38.381357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c4938-ce2f-4a6b-b8c7-028d80b84ef7 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.397497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2e83a7-59da-4a4d-b85c-44fa682bdbef None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.431703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.432364 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.432684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.432966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.433839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=9937f1088b304574935fcdf579e4de7d) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:38.434153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.435584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.449725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 330/678] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/role_assignments?user.id=9937f1088b304574935fcdf579e4de7d&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:38.449725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.494500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c4938-ce2f-4a6b-b8c7-028d80b84ef7 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.532727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.533489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.533775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.534081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.534855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:38.535207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.536817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.542560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 349/679] 10.222.0.35 () {56 vars in 1124 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:38.543494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.568890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72013e24-f683-4657-bbd2-6bc74a4b04db None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.579055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.585959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.610702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.611852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.613063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.613539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.621785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.623573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:38.641194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:38.642182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 331/680] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:38.644102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.669223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763b459e-4bed-4c1c-a65e-1d823336f4b0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.709969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.710765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.711046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.711324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.727678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:38.738283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:38.739262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:38.740121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 350/681] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:38.740923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.776734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606081a-1c9b-4c81-a1b6-69d8221aedac None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.830380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.831071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.831358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.831818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.833016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:38.833356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.835424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:38.843336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 332/682] 10.222.0.35 () {56 vars in 1124 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:38.844161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:38.855893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.888686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75dc4d-f273-4eed-a6c3-8dc292f770b6 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.891346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:38.974865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ee3c3-5a09-4c88-9582-5fea778ae9ed None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:38.993821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:38.994507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:38.994790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:38.995067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:38.995882 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:38.996190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:38.997696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:39.015632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:39.016718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:39.017210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:39.017980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:39.019036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:39.019795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:39.021495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:39.025654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 351/683] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:39.031575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:39.031887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 333/684] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:39.040762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:39.172785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab33ab6f-c67f-45a9-85ef-fe416f500f4c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:39.186657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:39.192459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7f83f-036c-4f45-98c3-0353da2de9c1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:39.211172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:39.212084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:39.212370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:39.212747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:39.213733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=9af9eae0fa5b437e96c1c732dfe4fa7a) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:39.214140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:39.215678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:39.223284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 352/685] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:39 2022] GET /identity/v3/role_assignments?user.id=9af9eae0fa5b437e96c1c732dfe4fa7a&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:39.248722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:39.249823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:39.250259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:39.250712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:39.263337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:39.265028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:39.297561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:39.298710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:39.300313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:39.300313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:39.300313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:39.300313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:39.302228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 334/686] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:39.308028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:39.339615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:39.466744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4902bc41-377f-4947-96a7-be9edb42b3c6 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:39.508562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:39.510026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:39.510318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:39.510598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:39.511332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:39.511635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:39.513019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:39.527253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 353/687] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:39.533426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:39.654499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4607-7ddb-4320-a99e-134201656378 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:39.693017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:39.693924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:39.694208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:39.694488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:39.702468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:39.703914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:39.719701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:39.721572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:39.723110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 335/688] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:39.731217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:39.818849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:39.845473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9185e6-7804-4cd0-9e21-08cae1030ce9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:39.887305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:39.888069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:39.888279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:39.888554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:39.889438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:39.889748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:39.891195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:39.910728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 354/689] 10.222.0.35 () {56 vars in 1135 bytes} [Tue May 24 09:53:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:39.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:39.932853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:40.049382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d46858-c509-4bfd-bcdf-322a9a1eed6f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:40.104357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfd359-a6a9-4a9e-8fd3-b57331f13bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:40.105020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfd359-a6a9-4a9e-8fd3-b57331f13bde None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:40.105327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfd359-a6a9-4a9e-8fd3-b57331f13bde None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:40.105610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfd359-a6a9-4a9e-8fd3-b57331f13bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:40.206681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfd359-a6a9-4a9e-8fd3-b57331f13bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:40.208259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfd359-a6a9-4a9e-8fd3-b57331f13bde None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:40.226403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:40.351328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 336/690] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 24 09:53:40.356392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:40.398136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eaa61-bf93-4db0-8f9f-9e95a2cd27c3 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:40.398458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eaa61-bf93-4db0-8f9f-9e95a2cd27c3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:40.398713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42eaa61-bf93-4db0-8f9f-9e95a2cd27c3 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:40.400341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 337/691] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:40.405609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a35a-8a61-4576-85fb-fdf84f87087f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:40.405904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a35a-8a61-4576-85fb-fdf84f87087f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:40.406188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8a35a-8a61-4576-85fb-fdf84f87087f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:40.443620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4fd8a35a-8a61-4576-85fb-fdf84f87087f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:40.444818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4fd8a35a-8a61-4576-85fb-fdf84f87087f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:40.548829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 338/692] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:40.553813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e8492-a914-4d9d-acc8-9ad456986712 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:40.584645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8a35a-8a61-4576-85fb-fdf84f87087f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:40.598117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:40.606994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:40.607731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:40.608014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:40.608287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:40.632911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:40.634409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:40.656535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:40.657638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:40.659287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:40.659287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:40.659287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:40.659287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:40.659287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 355/693] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:40.663264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:40.673942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:40.787306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfd245-f9a9-451f-a213-547429ffa9c5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:40.819541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8a35a-8a61-4576-85fb-fdf84f87087f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:40.829830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:40.830644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:40.831565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:40.831935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:40.832806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:40.837288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:40.837288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:40.848801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 356/694] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:40.853826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:40.902086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:40.902820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:40.903109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:40.903386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:40.904747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:40.934191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:40.950158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:40.951137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:40.952548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 339/695] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:40.958175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:40.962937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8630e80-defc-4e70-aeff-a905d927ffdf None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:40.998861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8524a520-3efa-4365-8846-f831facae2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:40.999689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524a520-3efa-4365-8846-f831facae2fc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:40.999979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524a520-3efa-4365-8846-f831facae2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.000256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8524a520-3efa-4365-8846-f831facae2fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.007939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524a520-3efa-4365-8846-f831facae2fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.009415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8524a520-3efa-4365-8846-f831facae2fc None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-8524a520-3efa-4365-8846-f831facae2fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:41.024947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:41.025940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:41.027221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 357/696] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:41.027949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a96fd-7c35-4dd9-bef9-bb86ffc903b2 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.028136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a96fd-7c35-4dd9-bef9-bb86ffc903b2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.028374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a96fd-7c35-4dd9-bef9-bb86ffc903b2 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.030944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 358/697] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:41.032727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a96fd-7c35-4dd9-bef9-bb86ffc903b2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.127734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:41.133255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fa2f1-cd00-43cd-9c92-17dcf32d6a9b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.149640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a96fd-7c35-4dd9-bef9-bb86ffc903b2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.153337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:41.185603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b193c-a040-43db-88d4-ceb23c33f4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.187145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b193c-a040-43db-88d4-ceb23c33f4bb None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.187582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b193c-a040-43db-88d4-ceb23c33f4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.188037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b193c-a040-43db-88d4-ceb23c33f4bb None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.189005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b193c-a040-43db-88d4-ceb23c33f4bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:41.189482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b193c-a040-43db-88d4-ceb23c33f4bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.191589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b193c-a040-43db-88d4-ceb23c33f4bb None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.192008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.192922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.193299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.193699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.194565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:41.194939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.196552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.203117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 340/698] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:41.205257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c1936-7288-436c-a278-2efebe263d0b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.205757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c1936-7288-436c-a278-2efebe263d0b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.207013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1c1936-7288-436c-a278-2efebe263d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.214880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 359/699] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:41.216403 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.230801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1b1c1936-7288-436c-a278-2efebe263d0b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:41.232327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1b1c1936-7288-436c-a278-2efebe263d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:41.301161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 341/700] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:41.302184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c1936-7288-436c-a278-2efebe263d0b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.345362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c884b-0a4c-46c6-9c16-26ec05602e9b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.385470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.386640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.387068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.387516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.400529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.402341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.403535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:41.415562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1c1936-7288-436c-a278-2efebe263d0b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:41.422715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:41.423593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:41.424504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:41.424504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:41.424504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:41.424504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:41.428685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 360/701] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:41.429716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.450247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:41.454295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.455075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.455461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.455806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.457035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=05f46d019e1348adabd023de481f0a3d) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:41.457442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.459051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.468210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 342/702] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/role_assignments?user.id=05f46d019e1348adabd023de481f0a3d&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:41.469089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.544337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88083c-5be0-483a-a4cc-5394e486c083 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.585737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c49a9fd-611d-40f9-b1af-1cacfa12487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.587044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a9fd-611d-40f9-b1af-1cacfa12487a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.587557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a9fd-611d-40f9-b1af-1cacfa12487a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.588083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c49a9fd-611d-40f9-b1af-1cacfa12487a None admin] PATH_INFO: `/v3/services/network` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.589084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49a9fd-611d-40f9-b1af-1cacfa12487a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.590756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c49a9fd-611d-40f9-b1af-1cacfa12487a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-4c49a9fd-611d-40f9-b1af-1cacfa12487a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:41.607199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:41.608273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:41.609131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:41.612553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 361/703] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:41.614402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253f87-f931-489f-ac1d-b4d367dac388 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.614922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253f87-f931-489f-ac1d-b4d367dac388 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.615403 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b253f87-f931-489f-ac1d-b4d367dac388 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.617463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 362/704] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:41.622615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253f87-f931-489f-ac1d-b4d367dac388 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.628935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405ec8a-f4ee-4ebf-98dd-7824a7fbdda1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.668362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedce0e-8a86-408b-9058-48750a470ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.668957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedce0e-8a86-408b-9058-48750a470ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.669234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedce0e-8a86-408b-9058-48750a470ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.669538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedce0e-8a86-408b-9058-48750a470ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.670281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedce0e-8a86-408b-9058-48750a470ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:41.670585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedce0e-8a86-408b-9058-48750a470ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.672003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedce0e-8a86-408b-9058-48750a470ff0 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.692174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 343/705] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:41.693800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740e9a-fe47-4f61-a0d1-3aa9fcc765f0 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.694134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740e9a-fe47-4f61-a0d1-3aa9fcc765f0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.694413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740e9a-fe47-4f61-a0d1-3aa9fcc765f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.712746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f7740e9a-fe47-4f61-a0d1-3aa9fcc765f0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:41.713962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f7740e9a-fe47-4f61-a0d1-3aa9fcc765f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:41.726000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:41.727354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:41.750299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b253f87-f931-489f-ac1d-b4d367dac388 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.797647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.798717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.798717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.798922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.799603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] RBAC: Adding query filter params (name=network) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:41.799966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.801514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.809597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 344/706] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:41.809597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740e9a-fe47-4f61-a0d1-3aa9fcc765f0 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.816707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 363/707] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:41.822478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:41.923785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740e9a-fe47-4f61-a0d1-3aa9fcc765f0 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.951549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e6f360-a557-4053-bc6f-14d84cacc227 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:41.960601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.961543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.961825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.962100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.971122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:41.973309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:41.992547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:41.993475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:41.993915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:41.994342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:41.994569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:41.995461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:41.996413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 345/708] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:41.998290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] RBAC: Adding query filter params (type=network) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:41.998819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:42.000385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:42.001281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:42.008895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 364/709] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:53:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:42.010102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:42.011774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:42.025708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:42.124709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f82cd-3562-49bc-af64-ed845a648d2e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:42.144773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba230d5-422e-4082-82a3-09096136988c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:42.188761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:42.189936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.190485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.190973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.192063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:42.192619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:42.194555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:42.214394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86858251-b53b-4c10-84df-3350d5f9f739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:42.215459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858251-b53b-4c10-84df-3350d5f9f739 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.215946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858251-b53b-4c10-84df-3350d5f9f739 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.221334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86858251-b53b-4c10-84df-3350d5f9f739 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.221334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86858251-b53b-4c10-84df-3350d5f9f739 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:42.221334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86858251-b53b-4c10-84df-3350d5f9f739 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:42.228007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 365/710] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:53:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:42.233190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:42.312053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:42.314425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 346/711] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:53:41 2022] POST /identity/v3/services => generated 251 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:42.362716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d9e8f-28fe-43d3-95f9-a95ad82a47de None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:42.374872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85f837-9055-412b-8296-02e5080aef35 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.375194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85f837-9055-412b-8296-02e5080aef35 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.375431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85f837-9055-412b-8296-02e5080aef35 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.378830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 347/712] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:42.383356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d23ac-1cd0-4f51-b36a-f01762b14340 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.383672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d23ac-1cd0-4f51-b36a-f01762b14340 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.383923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d23ac-1cd0-4f51-b36a-f01762b14340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.406538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-6d0d23ac-1cd0-4f51-b36a-f01762b14340 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:42.407787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6d0d23ac-1cd0-4f51-b36a-f01762b14340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:42.417237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:42.418995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.419529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.420513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.503404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:42.512685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 348/713] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 24 09:53:42.536346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d23ac-1cd0-4f51-b36a-f01762b14340 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:42.578669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:42.580713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:42.664726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:42.674529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d23ac-1cd0-4f51-b36a-f01762b14340 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:42.706190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 366/714] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:42 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 24 09:53:42.713297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:42.713837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:42.714631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.714910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.715181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.728486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:42.730181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:42.750657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:42.751632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:42.752624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:42.752624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:42.752624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:42.752624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:42.755565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 349/715] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:42.758708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:42.814958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:42.833551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b1dc4-bacc-4375-b3c4-c4b750acb239 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:42.881577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:42.882386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.882670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.882944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.894321 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:42.895952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:42.914964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:42.915997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:42.916956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:42.916956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:42.916956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:42.916956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:42.916956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 367/716] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:42.921882 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:42.933105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425d0f0-ffb3-4ec3-a570-3b6b8376dfc8 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:42.938211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:42.993622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:42.994297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:42.994703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:42.994981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:42.995812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:42.996120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:42.997506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:43.013551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 350/717] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:43.015418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:43.100868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b463189-458f-4ef1-ba73-8805b38c1d72 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:43.137859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:43.138525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:43.138905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:43.139181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:43.139907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:43.140209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:43.141600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:43.158682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 368/718] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:43.164242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:43.205024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af0cca-85df-417b-a541-c1c74dc5924c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:43.233726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:43.275300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:43.286112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:43.286785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:43.287067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:43.287344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:43.288054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:43.290381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:43.319481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898e58f-36a5-49cc-aff0-a72dcb9ae604 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:43.355409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:43.356249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:43.356529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:43.356801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:43.364281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:43.365888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:43.381200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:43.382124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:43.383009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 369/719] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:43.388050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:43.404416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 351/720] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:53:43 2022] POST /identity/v3/users => generated 254 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:53:43.505739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f7a28b-7d9f-49e9-a834-27aab180a23e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:43.568260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb96a0-61db-4078-866d-03579361a74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:43.569333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb96a0-61db-4078-866d-03579361a74f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:43.569819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb96a0-61db-4078-866d-03579361a74f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:43.570308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb96a0-61db-4078-866d-03579361a74f None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:43.571317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb96a0-61db-4078-866d-03579361a74f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:43.571827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb96a0-61db-4078-866d-03579361a74f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:43.573524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb96a0-61db-4078-866d-03579361a74f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:43.579366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:43.592032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 370/721] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:43.598601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:43.733369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fd51a-0c73-4d27-917b-baee683507bb None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:43.775283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e9fe7d-7587-445d-91a6-7764e7eeba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:43.776295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9fe7d-7587-445d-91a6-7764e7eeba69 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:43.776295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9fe7d-7587-445d-91a6-7764e7eeba69 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:43.776474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e9fe7d-7587-445d-91a6-7764e7eeba69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:43.777341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9fe7d-7587-445d-91a6-7764e7eeba69 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=8de75a1df6784111aabec60da40ab756) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:43.777658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9fe7d-7587-445d-91a6-7764e7eeba69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:43.779110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e9fe7d-7587-445d-91a6-7764e7eeba69 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:43.786573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 352/722] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:43 2022] GET /identity/v3/role_assignments?user.id=8de75a1df6784111aabec60da40ab756&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:44.193231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b7b99-ea38-4e85-9432-6fad2cb3a96e None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:44.194252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b7b99-ea38-4e85-9432-6fad2cb3a96e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:44.194756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b7b99-ea38-4e85-9432-6fad2cb3a96e None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:44.196583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 371/723] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:44.202231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f95a-25bf-463d-ba8e-6adb7f28d07f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:44.203052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f95a-25bf-463d-ba8e-6adb7f28d07f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:44.203550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76f95a-25bf-463d-ba8e-6adb7f28d07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:44.222744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-bb76f95a-25bf-463d-ba8e-6adb7f28d07f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:44.224434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bb76f95a-25bf-463d-ba8e-6adb7f28d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:44.250153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:44.324025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 353/724] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:44.364565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b7b99-ea38-4e85-9432-6fad2cb3a96e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:44.504186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b7b99-ea38-4e85-9432-6fad2cb3a96e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:44.559108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:44.559907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:44.560194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:44.560517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:44.576302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:44.585089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:44.586088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:44.587341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 372/725] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:44.592497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76f95a-25bf-463d-ba8e-6adb7f28d07f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:44.712149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76f95a-25bf-463d-ba8e-6adb7f28d07f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:44.748184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b10c3-f6cb-476c-a4c2-56507ce828ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:44.748863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b10c3-f6cb-476c-a4c2-56507ce828ee None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:44.749147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b10c3-f6cb-476c-a4c2-56507ce828ee None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:44.749459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b10c3-f6cb-476c-a4c2-56507ce828ee None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:44.750225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b10c3-f6cb-476c-a4c2-56507ce828ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:44.750594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b10c3-f6cb-476c-a4c2-56507ce828ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:44.751974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b10c3-f6cb-476c-a4c2-56507ce828ee None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:44.761923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 354/726] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:44.767335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:44.832050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56261a31-b2bc-41e9-aa53-ee55a865760f None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:44.832376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56261a31-b2bc-41e9-aa53-ee55a865760f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:44.832621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56261a31-b2bc-41e9-aa53-ee55a865760f None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:44.834025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 355/727] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:44.846123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684f7f8-b0cb-47fb-b99b-27da3f10586d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:44.846416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684f7f8-b0cb-47fb-b99b-27da3f10586d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:44.847251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684f7f8-b0cb-47fb-b99b-27da3f10586d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:44.850841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:44.880822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8684f7f8-b0cb-47fb-b99b-27da3f10586d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:44.883412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8684f7f8-b0cb-47fb-b99b-27da3f10586d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:44.916005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:44.944755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509446c-1d85-4ced-b230-3a2793c34fff None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:44.992911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:44.994140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:44.994621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:44.995085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.027973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:45.029424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:45.067818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:45.068800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:45.069760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:45.069760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:45.069760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:45.069760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:45.077967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 373/728] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:45.084288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:45.135888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 356/729] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 24 09:53:45.162053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684f7f8-b0cb-47fb-b99b-27da3f10586d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:45.272259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b36145-0256-4a01-a15d-183152f84a6c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:45.285692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:45.311075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f568616f-b971-4f8b-8141-766cdba2f74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:45.312213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568616f-b971-4f8b-8141-766cdba2f74b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.312706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568616f-b971-4f8b-8141-766cdba2f74b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.313188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568616f-b971-4f8b-8141-766cdba2f74b None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.314373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568616f-b971-4f8b-8141-766cdba2f74b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:45.315024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568616f-b971-4f8b-8141-766cdba2f74b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:45.317744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f568616f-b971-4f8b-8141-766cdba2f74b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:45.321563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684f7f8-b0cb-47fb-b99b-27da3f10586d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:45.331752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 374/730] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:45.337791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9df6c-53d8-406f-9dc6-5f884368faa8 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.338309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9df6c-53d8-406f-9dc6-5f884368faa8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.338830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9df6c-53d8-406f-9dc6-5f884368faa8 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.340734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 375/731] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:45.342078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9df6c-53d8-406f-9dc6-5f884368faa8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:45.365069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5deb897-5e6d-4fc2-acac-ea6ddccad967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:45.365767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deb897-5e6d-4fc2-acac-ea6ddccad967 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.366048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deb897-5e6d-4fc2-acac-ea6ddccad967 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.366322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5deb897-5e6d-4fc2-acac-ea6ddccad967 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.367214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5deb897-5e6d-4fc2-acac-ea6ddccad967 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:45.369354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5deb897-5e6d-4fc2-acac-ea6ddccad967 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:45.454160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 357/732] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:53:45 2022] POST /identity/v3/services => generated 240 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:53:45.454160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9ef5-e7d0-494d-a04d-5d3a66414b38 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.454160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9ef5-e7d0-494d-a04d-5d3a66414b38 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.454160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded9ef5-e7d0-494d-a04d-5d3a66414b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.472307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9df6c-53d8-406f-9dc6-5f884368faa8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:45.477578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-dded9ef5-e7d0-494d-a04d-5d3a66414b38 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:45.480218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-dded9ef5-e7d0-494d-a04d-5d3a66414b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:45.483531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:45.550787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:45.552080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.552596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.553090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] PATH_INFO: `/v3/users/swift` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.573949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 358/733] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:45.575088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:45.576756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:45.593172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded9ef5-e7d0-494d-a04d-5d3a66414b38 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:45.613299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:45.614576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:45.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:45.617762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 376/734] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:45.623380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:45.680289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:45.712205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded9ef5-e7d0-494d-a04d-5d3a66414b38 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:45.753740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:45.754934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.755444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.755932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.756711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:45.761289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:45.773359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:45.774485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:45.775341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:45.776156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 359/735] 10.222.0.35 () {56 vars in 1129 bytes} [Tue May 24 09:53:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:45.781965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:45.793696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:45.858566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d5f42-9370-4ee9-a50f-cbfdac956dce None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:45.903013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:45.904051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.904568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.905055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.906146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:45.906669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:45.908307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:45.922101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde0a99-1d16-42cb-8da0-2d769191aa33 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:45.929077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 377/736] 10.222.0.35 () {56 vars in 1135 bytes} [Tue May 24 09:53:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:45.934844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:45.966417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:45.967501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:45.967941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:45.968230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:45.969229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:45.969568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:45.972068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:45.978826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 360/737] 10.222.0.35 () {56 vars in 1124 bytes} [Tue May 24 09:53:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:45.985022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:46.027515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:46.047743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ad0f1-0483-4e7d-ae87-8b1b460398db None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.088364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e850b-3c78-4f96-8459-bd8544545aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.089476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e850b-3c78-4f96-8459-bd8544545aca None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.089973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e850b-3c78-4f96-8459-bd8544545aca None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.090454 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e850b-3c78-4f96-8459-bd8544545aca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.111596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:46.166697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e850b-3c78-4f96-8459-bd8544545aca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:46.168415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e850b-3c78-4f96-8459-bd8544545aca None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:46.193343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80edb4df-a408-4c6f-9e43-740d29b1a597 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.261177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b6af9-a273-47dd-a583-fafa198e14cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.261177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b6af9-a273-47dd-a583-fafa198e14cd None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.261177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b6af9-a273-47dd-a583-fafa198e14cd None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.261177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b6af9-a273-47dd-a583-fafa198e14cd None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.261661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b6af9-a273-47dd-a583-fafa198e14cd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:46.261661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b6af9-a273-47dd-a583-fafa198e14cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:46.261661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303b6af9-a273-47dd-a583-fafa198e14cd None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:46.266796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:46.266961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 361/738] 10.222.0.35 () {56 vars in 1124 bytes} [Tue May 24 09:53:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:46.274096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7785-5e8f-469a-922c-1658726cb18f None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.274517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7785-5e8f-469a-922c-1658726cb18f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.274902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7f7785-5e8f-469a-922c-1658726cb18f None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.276547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 362/739] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:46.277358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7785-5e8f-469a-922c-1658726cb18f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:46.292700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 378/740] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 09:53:46.298765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d7973-ccfa-46c3-bad7-6c65c1179e11 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.298765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d7973-ccfa-46c3-bad7-6c65c1179e11 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.298765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45d7973-ccfa-46c3-bad7-6c65c1179e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.320717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a45d7973-ccfa-46c3-bad7-6c65c1179e11 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:46.322301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a45d7973-ccfa-46c3-bad7-6c65c1179e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:46.398940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 379/741] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:46.400200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d7973-ccfa-46c3-bad7-6c65c1179e11 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:46.429637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7f7785-5e8f-469a-922c-1658726cb18f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.466252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:46.472690 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.473489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.474145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.476872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.477961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] RBAC: Adding query filter params (interface=public, service_id=9e849706a2db49589f76f7091bf16bc1, region_id=RegionOne) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:46.478513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:46.480102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:46.490167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 363/742] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:53:46 2022] GET /identity/v3/endpoints?service_id=9e849706a2db49589f76f7091bf16bc1&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:46.494343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:46.514862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45d7973-ccfa-46c3-bad7-6c65c1179e11 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.526117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:46.550705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.551570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.551855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.552133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.563711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:46.565170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:46.584303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:46.585353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:46.586288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:46.586288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:46.586288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:46.586288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:46.590180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 380/743] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:46.591187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:46.627140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f254c3a-2e12-4301-bcb8-2849ac6562cf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.677630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.678661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.679110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.679561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.694732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:46.706140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:46.707341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:46.708390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:46.715503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 364/744] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:46.715503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:46.717595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a9e33-7841-4934-b9b7-4ad310ef5e67 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.755029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.756687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.757177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.757658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.759100 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:46.760930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:46.768729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 381/745] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:53:46 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:46.769925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:46.787339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:46.818706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:46.848888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e38201-4195-463d-abc4-fc620fee7f15 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.889214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e47e3-ef17-407f-a2ff-646d0cd4aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.890363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e47e3-ef17-407f-a2ff-646d0cd4aafe None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.890879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e47e3-ef17-407f-a2ff-646d0cd4aafe None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.891407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e47e3-ef17-407f-a2ff-646d0cd4aafe None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.892356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e47e3-ef17-407f-a2ff-646d0cd4aafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:46.893129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e47e3-ef17-407f-a2ff-646d0cd4aafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:46.895622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e47e3-ef17-407f-a2ff-646d0cd4aafe None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:46.909041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 365/746] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:46.910880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22944a-1b39-4f11-899f-59c16239a70d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:46.911653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545c46-7149-47df-ace8-62ebf219f3a1 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.912169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545c46-7149-47df-ace8-62ebf219f3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.912650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7545c46-7149-47df-ace8-62ebf219f3a1 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.914706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 366/747] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:46.922353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d827960-9f31-4e2b-942d-dbe17f33a88d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.923059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d827960-9f31-4e2b-942d-dbe17f33a88d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.923569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d827960-9f31-4e2b-942d-dbe17f33a88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.942608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1d827960-9f31-4e2b-942d-dbe17f33a88d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:46.944205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1d827960-9f31-4e2b-942d-dbe17f33a88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:46.952331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:46.953027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:46.953375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:46.953646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:46.954425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:46.954735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:46.956734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:46.972318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 382/748] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:46.973432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.023894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 367/749] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:47.024761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d827960-9f31-4e2b-942d-dbe17f33a88d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.069095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:47.076789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:47.083785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023ec16-42b8-4455-94a3-7ab868d7fb0c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.120779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.121615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.121900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.122264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.130031 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.131508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.133375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d827960-9f31-4e2b-942d-dbe17f33a88d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:47.146715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:47.147752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:47.148662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 383/750] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:47.149156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.187497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.188682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.189197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.189786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.202023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.203950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:47.225004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:47.226008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:47.226957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:47.226957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:47.226957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:47.226957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:47.229157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 368/751] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:47.230229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.269764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b396b1-5c3b-4575-bd83-312140a89881 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.308042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.308739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.308739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.308907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.321301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:47.329900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:47.331142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:47.332525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 384/752] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:47.333483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.341536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:47.345923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:47.346666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf317c4-e428-4acd-839b-dc65b2e43e38 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.383010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.383980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.384447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.385004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.386027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:47.386520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.388099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.406183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 369/753] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:47.407301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.446093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60826e42-aeff-4c92-a532-10d087a6ac13 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.485233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.485912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.486206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.486635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.489018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:47.489018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.490051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.503931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 385/754] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:47.509546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.522549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f506bd0c-cf83-4960-89b6-f294baa98295 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.559537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.560665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.561136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.561689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.562630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:47.563270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.566405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.577643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 370/755] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:47.579514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.584686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:47.606071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:47.624012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453c76a-3070-4d99-a2ba-72832720480f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.664021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245157-478c-4710-b18a-888caa53352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.665004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245157-478c-4710-b18a-888caa53352d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.665537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245157-478c-4710-b18a-888caa53352d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.666025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be245157-478c-4710-b18a-888caa53352d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.667164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245157-478c-4710-b18a-888caa53352d None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=05f46d019e1348adabd023de481f0a3d) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:47.667678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245157-478c-4710-b18a-888caa53352d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.669448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be245157-478c-4710-b18a-888caa53352d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.677846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 386/756] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/role_assignments?user.id=05f46d019e1348adabd023de481f0a3d&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:47.678882 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245157-478c-4710-b18a-888caa53352d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.724549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5bbc3-322d-4e3e-9da8-d7ef97ac8728 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.799528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.800720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.801034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.801366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.810693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.812145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:47.843045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:47.844070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:47.845021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 371/757] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:47.856149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.858980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be245157-478c-4710-b18a-888caa53352d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:47.904536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:47.911830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:47.917096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:47.917096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:47.917096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:47.931620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:47.933102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:47.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:47.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:47.975649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:47.975649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:47.975649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:47.975649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:47.975649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 387/758] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:47.980808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:47.995904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:48.037540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4afa0c-38f2-4b4e-84f5-4a4cc8bc2a8a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.124520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.125118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.125436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.125715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.127134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:48.127456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.129493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.163063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 372/759] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:48.167978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:48.201880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3d70a-aa65-4a8f-ad2d-5a41d4284c2a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.259373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f87a2-be7d-4f69-a40c-ab88c02918e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.260475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f87a2-be7d-4f69-a40c-ab88c02918e9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.261033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f87a2-be7d-4f69-a40c-ab88c02918e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.261524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f87a2-be7d-4f69-a40c-ab88c02918e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.262450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f87a2-be7d-4f69-a40c-ab88c02918e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:48.262966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f87a2-be7d-4f69-a40c-ab88c02918e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.264533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f87a2-be7d-4f69-a40c-ab88c02918e9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.278533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 388/760] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:48.280220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940c82c-83e3-474b-9d3f-bf4c9e85b9ee None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.280709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940c82c-83e3-474b-9d3f-bf4c9e85b9ee None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.281178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940c82c-83e3-474b-9d3f-bf4c9e85b9ee None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.282872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 389/761] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:48.283771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940c82c-83e3-474b-9d3f-bf4c9e85b9ee None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:48.322219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b2171-bc2e-48d2-990d-85143622799a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.333491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:48.359980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dade4-3dd8-4d5e-91ed-d25342a75c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.360466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dade4-3dd8-4d5e-91ed-d25342a75c36 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.360760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dade4-3dd8-4d5e-91ed-d25342a75c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.361049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dade4-3dd8-4d5e-91ed-d25342a75c36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.370124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:48.375383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dade4-3dd8-4d5e-91ed-d25342a75c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.377237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dade4-3dd8-4d5e-91ed-d25342a75c36 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.394386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940c82c-83e3-474b-9d3f-bf4c9e85b9ee None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-e11dade4-3dd8-4d5e-91ed-d25342a75c36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:48.398549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:48.399537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:48.400454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:48.400454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:48.400454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:48.400454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:48.407913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 373/762] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:48.410412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150d306-38af-464f-8709-9b473c8f2736 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.411107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150d306-38af-464f-8709-9b473c8f2736 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.411619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150d306-38af-464f-8709-9b473c8f2736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.430183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9150d306-38af-464f-8709-9b473c8f2736 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:48.431665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9150d306-38af-464f-8709-9b473c8f2736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:48.432285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.433180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.433569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.433931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.441686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.442904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:48.458985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:48.460005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:48.461003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 390/763] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:48.461652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:48.525461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 374/764] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:48.527249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150d306-38af-464f-8709-9b473c8f2736 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:48.586488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278cf4a3-fb47-4d22-a372-4fd74ae16883 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.590058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:48.624523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.625519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.625854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.626246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.627113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:48.627573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.629454 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.645918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 391/765] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:48.645918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:48.670315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150d306-38af-464f-8709-9b473c8f2736 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.702205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:48.722674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.723792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.724523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.725920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.726999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:48.727495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.729144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.755030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 375/766] 10.222.0.35 () {56 vars in 1149 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:48.756493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:48.761325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8949bd-cd44-4c5d-9ce3-3c39a8463041 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.801020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.801945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.802302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.802656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] PATH_INFO: `/v3/services/network` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.803680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.805151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:48.818301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:48.819337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:48.820372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 392/767] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:48.822331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:48.853230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:48.895675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00233638-cdcb-4101-ad46-c2b4f313c5aa None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.931942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:48.932911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:48.933303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:48.933692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:48.940981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:48.942682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:48.946259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c204f95-cf66-41b5-8af0-678de93e20de None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:48.958142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:48.959315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:48.960436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:48.966771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 376/768] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:48.966771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.006801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.008093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.008581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.009103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.015903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:49.091504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa585fd-2243-4832-98db-1381b238c6f8 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.103941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.105598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.128730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.129797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.130332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.130789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.131677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] RBAC: Adding query filter params (name=network) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:49.132167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.133828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.139070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 377/769] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:49.140064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.149543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:49.239672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 393/770] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:53:49.241447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.255114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f968c-108d-4836-866a-f9b59cb536fe None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.256903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:49.290908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9069c1-80dc-4144-8aad-6fdac216c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.291704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9069c1-80dc-4144-8aad-6fdac216c18c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.291951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9069c1-80dc-4144-8aad-6fdac216c18c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.292226 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9069c1-80dc-4144-8aad-6fdac216c18c None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.293124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9069c1-80dc-4144-8aad-6fdac216c18c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:49.293461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9069c1-80dc-4144-8aad-6fdac216c18c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.294954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9069c1-80dc-4144-8aad-6fdac216c18c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.313065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 378/771] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:49.313065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88995530-5a64-48c9-a1d9-9ddf9ceef8d9 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.313065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88995530-5a64-48c9-a1d9-9ddf9ceef8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.313065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88995530-5a64-48c9-a1d9-9ddf9ceef8d9 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.313065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 379/772] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:49.313065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88995530-5a64-48c9-a1d9-9ddf9ceef8d9 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.349586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8481f-0dac-4637-b672-ce15d4064fdc None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.375420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:49.385000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.385695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.385979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.386844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.387315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:49.387315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.388631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.398960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 394/773] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:49.398960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.423023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88995530-5a64-48c9-a1d9-9ddf9ceef8d9 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.460991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd1254-663c-450e-8291-020a4f6126e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.461879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd1254-663c-450e-8291-020a4f6126e5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.462162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd1254-663c-450e-8291-020a4f6126e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.462454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd1254-663c-450e-8291-020a4f6126e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.473705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd1254-663c-450e-8291-020a4f6126e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.475150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd1254-663c-450e-8291-020a4f6126e5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-1fbd1254-663c-450e-8291-020a4f6126e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:49.492371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:49.493327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:49.494221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:49.494221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:49.494221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:49.494221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:49.494221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 380/774] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:49.495015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d8f39-04ed-441a-80a6-05343d6292f6 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.495015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d8f39-04ed-441a-80a6-05343d6292f6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.495380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d8f39-04ed-441a-80a6-05343d6292f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.509032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c366094-b301-4e90-82b9-640e5d58e4a2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.512804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a25d8f39-04ed-441a-80a6-05343d6292f6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:49.514111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a25d8f39-04ed-441a-80a6-05343d6292f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:49.525366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:49.546842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.547920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.548316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.548687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/8de75a1df6784111aabec60da40ab756/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.551218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=8de75a1df6784111aabec60da40ab756, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.552815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.584869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 381/775] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:49.585906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d8f39-04ed-441a-80a6-05343d6292f6 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.628866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 395/776] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:49 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/8de75a1df6784111aabec60da40ab756/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:49.631190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.711954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:49.715936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d8f39-04ed-441a-80a6-05343d6292f6 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.767149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.768533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.769112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.770008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.771392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] RBAC: Adding query filter params (interface=public, service_id=f88c6f3b8618420bbbf6eb28ee3a31f2, region_id=RegionOne) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:49.771818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.773988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.776499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac0317b-a1fe-47f9-aefa-63f89a8e9344 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.782651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 382/777] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/endpoints?service_id=f88c6f3b8618420bbbf6eb28ee3a31f2&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:49.786349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.802950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:49.831032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.832131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.832680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.832963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.833736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:49.834045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.836401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.851443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 396/778] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:49.852205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:49.919338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b29bd-cd74-4993-bf1a-e30c584fe4ca None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.968214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:49.968944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:49.969223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:49.970168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:49.971148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:49.972731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:49.980360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae03e1-4a54-4443-849c-8a0c642d9e52 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:49.992405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:49.993493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:49.994818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 383/779] 10.222.0.35 () {56 vars in 1129 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:49.998158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:50.000235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:50.023301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.024119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.024555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.024999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.025755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:50.027745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:50.033847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 397/780] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:50.038333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:50.088340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:50.127290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d554bf95-4a29-4cea-b539-b65879827f13 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:50.179293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92e183-cc01-410b-a9d1-c4d76eb44045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.180422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92e183-cc01-410b-a9d1-c4d76eb44045 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.180708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92e183-cc01-410b-a9d1-c4d76eb44045 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.180987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92e183-cc01-410b-a9d1-c4d76eb44045 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.191910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92e183-cc01-410b-a9d1-c4d76eb44045 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:50.193632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92e183-cc01-410b-a9d1-c4d76eb44045 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:50.203839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20114b-d7d6-4f3b-bd15-f5f5c0d0747c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-eb92e183-cc01-410b-a9d1-c4d76eb44045 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:50.212434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:50.217804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 384/781] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2a138-3cbd-4357-9850-574cd4662fb6 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.219047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2a138-3cbd-4357-9850-574cd4662fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.219788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2a138-3cbd-4357-9850-574cd4662fb6 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.221351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 385/782] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:50.223077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2a138-3cbd-4357-9850-574cd4662fb6 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:50.274040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981f9a02-49fd-4194-ba66-374ab46a67b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.274720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f9a02-49fd-4194-ba66-374ab46a67b1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.275001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f9a02-49fd-4194-ba66-374ab46a67b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.275272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981f9a02-49fd-4194-ba66-374ab46a67b1 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.276002 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f9a02-49fd-4194-ba66-374ab46a67b1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:50.276306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f9a02-49fd-4194-ba66-374ab46a67b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:50.278319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981f9a02-49fd-4194-ba66-374ab46a67b1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:50.284960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 398/783] 10.222.0.35 () {56 vars in 1124 bytes} [Tue May 24 09:53:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:50.286984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4d172-9d36-414e-80dc-e91dcdf7fffe None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.287272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4d172-9d36-414e-80dc-e91dcdf7fffe None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.287555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4d172-9d36-414e-80dc-e91dcdf7fffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.306869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-bcf4d172-9d36-414e-80dc-e91dcdf7fffe None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:50.308191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bcf4d172-9d36-414e-80dc-e91dcdf7fffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:50.353738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2a138-3cbd-4357-9850-574cd4662fb6 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:50.358088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:50.388635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:50.395382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.396067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.396353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.397193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.398213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:50.398668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:50.400277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:50.406671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 399/784] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:50.408368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4d172-9d36-414e-80dc-e91dcdf7fffe None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:50.422820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 386/785] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:50.432002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:50.541976 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4d172-9d36-414e-80dc-e91dcdf7fffe None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:50.553159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982beb1-c628-45a7-8248-c5bad409c5da None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:50.584617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.585307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.585633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.586331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.587196 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:50.587619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:50.589010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:50.591880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.593107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.593700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.594181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.595294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=70ebdb4e82064aad89b0dcec0a74cad2) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:50.595816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:50.597960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:50.598835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 400/786] 10.222.0.35 () {56 vars in 1124 bytes} [Tue May 24 09:53:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:50.599729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:50.605746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 387/787] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:50 2022] GET /identity/v3/role_assignments?user.id=70ebdb4e82064aad89b0dcec0a74cad2&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:50.606750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:50.673724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:50.722480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:50.807672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52405195-1c26-45f9-a060-6b428a9b78ec None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:50.811473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3673a46-e38d-4510-ac7e-f31be5ab59e8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:50.884594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.885280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.885572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.885702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:50.885885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.886507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:50.886615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:50.886791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:50.887074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:50.887961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:50.910706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:50.919510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:50.924032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:50.925846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 401/788] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:50.929057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:51.017434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 388/789] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:53:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:51.032877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:51.048047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7d76dd-20c3-4475-8776-df0474ff8823 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:51.108199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:51.108797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:51.109081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:51.109389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:51.110390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:51.110788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:51.112814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:51.129872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 402/790] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:51.135819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:51.285119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc10e360-1bc1-4e86-b45d-ceffe733fffb None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:51.299262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:51.323081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:51.324046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:51.324588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:51.325066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:51.338337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:51.339807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:51.358493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:51.360773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:51.363870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:51.363870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:51.363870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:51.363870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:51.363870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 389/791] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:51.366278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:51.475922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09954d2-4688-4b57-9c4c-f90f0251e1b1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:51.512355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:51.513421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:51.513917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:51.514415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:51.515329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:51.515812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:51.517384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:51.531611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 403/792] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:51.537065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:51.672217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2489-45aa-48d3-b9a0-ad9a8bde853d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:51.712925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:51.714078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:51.714571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:51.715038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:51.726383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:51.728069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:51.748103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:51.749144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:51.750145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:51.752471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 390/793] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:51.757988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:51.778141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:51.873811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678b835-13b1-42eb-b3fa-8aef2af50b31 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:51.911320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442c335-a003-4f26-b52a-ad0caef6e19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:51.912271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c335-a003-4f26-b52a-ad0caef6e19c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:51.912704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c335-a003-4f26-b52a-ad0caef6e19c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:51.913128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442c335-a003-4f26-b52a-ad0caef6e19c None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:51.914156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442c335-a003-4f26-b52a-ad0caef6e19c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:51.914631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442c335-a003-4f26-b52a-ad0caef6e19c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:51.916223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442c335-a003-4f26-b52a-ad0caef6e19c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:51.935541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 404/794] 10.222.0.35 () {56 vars in 1135 bytes} [Tue May 24 09:53:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:51.942536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:51.974984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:52.076946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77de1cf-b386-4c7a-8875-251a3fea67da None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:52.123326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:52.124130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.124413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.124689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.138438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:52.146318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:52.173216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e104c-68fa-4195-9cfa-d4c8bd562f7b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.173744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e104c-68fa-4195-9cfa-d4c8bd562f7b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.174033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e104c-68fa-4195-9cfa-d4c8bd562f7b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.175779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 405/795] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:52.181138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138dbc3-ba5e-4e7d-80ef-047cb3365872 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.181450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138dbc3-ba5e-4e7d-80ef-047cb3365872 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.181726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138dbc3-ba5e-4e7d-80ef-047cb3365872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:52.194459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:52.195421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:52.196379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:52.196379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:52.196379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:52.196379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:52.202000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 391/796] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:52.202895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:52.206421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4138dbc3-ba5e-4e7d-80ef-047cb3365872 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:52.207947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4138dbc3-ba5e-4e7d-80ef-047cb3365872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:52.310823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:52.335629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6506ad-1b42-43b7-beba-4cd2a98e4683 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:52.354546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:52.373697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 406/797] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:53:52.376084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:52.377077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.377615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.378739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.379688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:52.380173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:52.384219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:52.395809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 392/798] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:52.401354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138dbc3-ba5e-4e7d-80ef-047cb3365872 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:52.414427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:52.521643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dc045-b7e6-4ad2-844e-9010e231b239 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:52.557339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:52.558303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.558682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.559060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.564162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:52.570553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:52.578701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:52.587352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:52.588221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:52.588865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138dbc3-ba5e-4e7d-80ef-047cb3365872 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:52.588941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 393/799] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:52.588941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:52.649432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5397f8-0a8e-4e93-8130-22879f0d5985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:52.650553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5397f8-0a8e-4e93-8130-22879f0d5985 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.651021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5397f8-0a8e-4e93-8130-22879f0d5985 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.651478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5397f8-0a8e-4e93-8130-22879f0d5985 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.662644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5397f8-0a8e-4e93-8130-22879f0d5985 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:52.664325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5397f8-0a8e-4e93-8130-22879f0d5985 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9d5397f8-0a8e-4e93-8130-22879f0d5985 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:52.685212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:52.686248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:52.687154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:52.690247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 407/800] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:52.692211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91933b3f-fde1-40e5-ac99-663062f584f4 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.692679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91933b3f-fde1-40e5-ac99-663062f584f4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.693127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91933b3f-fde1-40e5-ac99-663062f584f4 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.694925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 408/801] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:52.702177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91933b3f-fde1-40e5-ac99-663062f584f4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:52.728591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff194624-db70-4a94-8c2a-de0f54a15892 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:52.768279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:52.784883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bc4f7d-6f5b-46d6-ba2e-3a5108c44d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:52.784883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc4f7d-6f5b-46d6-ba2e-3a5108c44d99 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.784883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc4f7d-6f5b-46d6-ba2e-3a5108c44d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.784883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bc4f7d-6f5b-46d6-ba2e-3a5108c44d99 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.785541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc4f7d-6f5b-46d6-ba2e-3a5108c44d99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:52.785541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc4f7d-6f5b-46d6-ba2e-3a5108c44d99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:52.786562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bc4f7d-6f5b-46d6-ba2e-3a5108c44d99 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:52.806049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 394/802] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:52.806049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7014e-bf56-499c-8e91-0dce3e82bf92 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.806049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7014e-bf56-499c-8e91-0dce3e82bf92 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.806049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7014e-bf56-499c-8e91-0dce3e82bf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.826532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91933b3f-fde1-40e5-ac99-663062f584f4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:52.830262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-dab7014e-bf56-499c-8e91-0dce3e82bf92 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:52.832492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-dab7014e-bf56-499c-8e91-0dce3e82bf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:52.868544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:52.869730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:52.870189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:52.871032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:52.872250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:52.872943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:52.875305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:52.892514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 409/803] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:52.894657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:52.900754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:52.916010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 395/804] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:52.916830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7014e-bf56-499c-8e91-0dce3e82bf92 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.023544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37afd50-5736-4f0a-8519-0c0cccf81f46 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.036136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7014e-bf56-499c-8e91-0dce3e82bf92 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.055751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:53.059510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.060306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.060629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.060901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.073542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.073715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.074560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.074916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.075095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.075309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/05f46d019e1348adabd023de481f0a3d/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.077870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=05f46d019e1348adabd023de481f0a3d, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.079457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:53.093309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:53.100427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:53.101384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:53.101384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:53.101384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:53.101384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:53.101384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 410/805] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:53.101384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.157922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 396/806] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:52 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/05f46d019e1348adabd023de481f0a3d/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:53.159470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.237128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4710e-d7f2-4fb2-9601-8e8a7ea9fbab None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.285776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:53.290461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.291580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.292076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.292566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] PATH_INFO: `/v3/services/network` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.293479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.295230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:53.319295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:53.320376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 24 09:53:53.321250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:53.331257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 411/807] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:53 2022] GET /identity/v3/services/network => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:53.338380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.356261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e241c-b5ac-458d-9323-06d17c6c0c1c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.396629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63e73b-eaa0-4260-8bde-b15a9d0231fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.397321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63e73b-eaa0-4260-8bde-b15a9d0231fa None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.397608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63e73b-eaa0-4260-8bde-b15a9d0231fa None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.397886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63e73b-eaa0-4260-8bde-b15a9d0231fa None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.398621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63e73b-eaa0-4260-8bde-b15a9d0231fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:53.398926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63e73b-eaa0-4260-8bde-b15a9d0231fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.400297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63e73b-eaa0-4260-8bde-b15a9d0231fa None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.405674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:53.414828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 397/808] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:53.416191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3909eb6-b132-401f-a63b-c18c62a816a3 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.416515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3909eb6-b132-401f-a63b-c18c62a816a3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.424355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3909eb6-b132-401f-a63b-c18c62a816a3 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.424355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 398/809] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:53.424355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3909eb6-b132-401f-a63b-c18c62a816a3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.480583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a43201-9055-4225-af48-1b6ce52d3544 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.516181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4cc5e-91a0-421f-8281-e1aaba7aad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.516855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4cc5e-91a0-421f-8281-e1aaba7aad3c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.517137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4cc5e-91a0-421f-8281-e1aaba7aad3c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.517445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4cc5e-91a0-421f-8281-e1aaba7aad3c None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.518935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4cc5e-91a0-421f-8281-e1aaba7aad3c None admin] RBAC: Adding query filter params (name=network) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:53.519312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4cc5e-91a0-421f-8281-e1aaba7aad3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.520958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4cc5e-91a0-421f-8281-e1aaba7aad3c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.527575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 412/810] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:53:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:53.528970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae156-0383-464a-84bc-bfcbbc605991 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.529259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae156-0383-464a-84bc-bfcbbc605991 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.529560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636ae156-0383-464a-84bc-bfcbbc605991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.548004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-636ae156-0383-464a-84bc-bfcbbc605991 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:53.549294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-636ae156-0383-464a-84bc-bfcbbc605991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:53.558886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3909eb6-b132-401f-a63b-c18c62a816a3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.570839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:53.603025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.604197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.604563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.604917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.616158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.618553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.637291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 413/811] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:53.637291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae156-0383-464a-84bc-bfcbbc605991 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:53.642306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:53.644265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:53:53.645446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:53.653688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 399/812] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:53:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:53.658853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.668501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:53.757191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636ae156-0383-464a-84bc-bfcbbc605991 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.781356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0cdf3b-313f-4876-a972-e40b91159da2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.800758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.801380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.801667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.801950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.802829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] RBAC: Adding query filter params (type=network) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:53.803435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.804816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.811520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 414/813] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:53:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:53.812347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.828581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.829619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.829972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.830318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.832519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:53.832913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:53.834837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:53.856926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 400/814] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:53:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:53.857879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:53.886432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:53.932981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:53.941978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b0814-c14f-4b87-9bfd-33c795d4e241 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.967068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816f90f-713b-4734-a5fc-2fc17141e3c0 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:53.981652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:53.982926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:53.983455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:53.983961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:53.996015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:54.004898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:54.005389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:54.006316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:54.007200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:54.007850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.007850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.007850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.007850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.008587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.010518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 415/815] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:54.011663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.117907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c653e-48e7-474b-b8ce-925050d0a447 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.120411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 401/816] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:53:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:54.130445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.174003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3402a0-e79d-4f4e-9665-771603d1ee93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.175218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3402a0-e79d-4f4e-9665-771603d1ee93 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.175711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3402a0-e79d-4f4e-9665-771603d1ee93 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.175996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3402a0-e79d-4f4e-9665-771603d1ee93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.198762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:54.258512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3402a0-e79d-4f4e-9665-771603d1ee93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.260022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3402a0-e79d-4f4e-9665-771603d1ee93 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.261630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:54.284327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06027bae-89e6-4a3c-9a64-9a83a1f05999 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.326126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d497b-903a-4da9-a5a7-59313b89405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.327141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d497b-903a-4da9-a5a7-59313b89405f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.327614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d497b-903a-4da9-a5a7-59313b89405f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.328059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d497b-903a-4da9-a5a7-59313b89405f None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.329023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d497b-903a-4da9-a5a7-59313b89405f None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:54.329539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d497b-903a-4da9-a5a7-59313b89405f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.331634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d497b-903a-4da9-a5a7-59313b89405f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.341028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 402/817] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:54.342691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f113f8-2380-436a-945c-b8fca1165056 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.343272 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f113f8-2380-436a-945c-b8fca1165056 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.343748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f113f8-2380-436a-945c-b8fca1165056 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.349378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 403/818] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:54.350212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f113f8-2380-436a-945c-b8fca1165056 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.384018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 416/819] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:53:54.385717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb86da-0a5d-4a5d-b6a2-31aaccf979b4 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.386259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb86da-0a5d-4a5d-b6a2-31aaccf979b4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.386775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb86da-0a5d-4a5d-b6a2-31aaccf979b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.409846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-52eb86da-0a5d-4a5d-b6a2-31aaccf979b4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:54.411708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-52eb86da-0a5d-4a5d-b6a2-31aaccf979b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:54.461713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:54.479844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f113f8-2380-436a-945c-b8fca1165056 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.482672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 417/820] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:54.483688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb86da-0a5d-4a5d-b6a2-31aaccf979b4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.530114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.530922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.531205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.531485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.543525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.544971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.548354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:54.564519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:54.566267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:54.567184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:54.567184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:54.567184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:54.567184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:54.567184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 404/821] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:54.567479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.593406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb86da-0a5d-4a5d-b6a2-31aaccf979b4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.630247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.631167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.631453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.631732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.642585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.644732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:54.664691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:54.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:54.666938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:54.666938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:54.666938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:54.666938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:54.670621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 418/822] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:54.671460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.678546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc43f5c-ec1f-495e-8cb8-9f1825afc823 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.705465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:54.712624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.713457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.713744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.717420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.717420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.717420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:54.728576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:54.729806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:53:54.730668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:54.731938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 405/823] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:54.733141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.780064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17828b41-acb9-4517-adf5-40417846e710 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.796862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:54.817420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.818101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.818385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.818663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.819394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:54.819702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.821166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.838323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 419/824] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:54.838323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.844706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb6ce5-5519-4147-ab7d-0c797811d1a5 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.880366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.881020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.881327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.881602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.882315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:54.882628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.883977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.897237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 406/825] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:54.897860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:54.937096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:54.946855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4b1748-744f-4ef8-84ed-90f997daf0b8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:54.983695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:54.984353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:54.984641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:54.984927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:54.985671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:54.985981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:54.987923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:54.992788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 420/826] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:54.993881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.007532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67898f-2be3-4889-9067-14ac162f7a5c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.028778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:55.045011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.045847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.046127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.046403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.054715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.056323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:55.073064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:55.074149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 407/827] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:55.080130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.108250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a409fdb-5ab4-4aa4-b69d-c0a1191503d3 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.150802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.151561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.151850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.152129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.162582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.164074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:55.179118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:55.180243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 421/828] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:55.181171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.193036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef906b0-c3a3-4298-b68b-5c8faf4f84c3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.202888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:55.243582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.244308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.244596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.244985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.245742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:55.246049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.247424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.261300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 408/829] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:55.261300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.320050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75dfa7-e7ed-4df0-b559-a683716fa8fb None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.329724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:55.372254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.373075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.373470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.373828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.375259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:55.375577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.377580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.404607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 422/830] 10.222.0.35 () {56 vars in 1149 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:55.408202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99cbc4b-3823-4164-9534-eb43ece6de41 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.411105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.455517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.456569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.457033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.457499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.458451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:55.458915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.460516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.475839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 409/831] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:55.481551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.514498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:55.541599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4405ddc-f026-4386-ae19-d37682e1835a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.586860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd635ccc-6411-4875-92f9-efce997a8ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.587765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd635ccc-6411-4875-92f9-efce997a8ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.588087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd635ccc-6411-4875-92f9-efce997a8ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.588332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd635ccc-6411-4875-92f9-efce997a8ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.596121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e669a-1b5e-4f9b-a0d9-4d0b8b5d8259 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.598861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:55.599809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd635ccc-6411-4875-92f9-efce997a8ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.601360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd635ccc-6411-4875-92f9-efce997a8ad8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-bd635ccc-6411-4875-92f9-efce997a8ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:55.620071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:55.625516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 423/832] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72b5f-5d0d-4797-a813-36327f0e35b4 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72b5f-5d0d-4797-a813-36327f0e35b4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c72b5f-5d0d-4797-a813-36327f0e35b4 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.626438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 424/833] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:55.630910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c72b5f-5d0d-4797-a813-36327f0e35b4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.636950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa16033-6208-4c57-91a7-4ee4886da82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.637746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa16033-6208-4c57-91a7-4ee4886da82a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.638114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa16033-6208-4c57-91a7-4ee4886da82a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.638437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa16033-6208-4c57-91a7-4ee4886da82a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.639400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa16033-6208-4c57-91a7-4ee4886da82a None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=8de75a1df6784111aabec60da40ab756) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:55.639775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa16033-6208-4c57-91a7-4ee4886da82a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.641813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa16033-6208-4c57-91a7-4ee4886da82a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.653417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 410/834] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/role_assignments?user.id=8de75a1df6784111aabec60da40ab756&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:55.654992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2971a4-2404-4b06-a4f7-2faa77a32c86 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.655436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2971a4-2404-4b06-a4f7-2faa77a32c86 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.655883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2971a4-2404-4b06-a4f7-2faa77a32c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.675748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0a2971a4-2404-4b06-a4f7-2faa77a32c86 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:55.677440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0a2971a4-2404-4b06-a4f7-2faa77a32c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:55.750595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c72b5f-5d0d-4797-a813-36327f0e35b4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.765698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 411/835] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:55.781122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:55.787133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2971a4-2404-4b06-a4f7-2faa77a32c86 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.792202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.793033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.793407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.797088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.797088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:55.797088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:55.797088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:55.811654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 425/836] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:55.820009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:55.916359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:55.919765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2971a4-2404-4b06-a4f7-2faa77a32c86 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.954607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d1139-6642-4382-9507-23e5d635dd61 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:55.972212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.973805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.974194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.974569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.988625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:55.991235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:55.992035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:55.992315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:55.992591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:55.997468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:55.998641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:53:55.999516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:56.000177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.000576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 412/837] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:56.005293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.007240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:56.019728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:56.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:56.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 426/838] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:56.027122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:56.094393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:56.125061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054534e2-b8c8-4a62-a278-bc1351a72e4f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:56.171361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:56.171808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc063f6-fd31-4995-9ce5-2a401e527754 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:56.172003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:56.172255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:56.172530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:56.173305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:56.173630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.175679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.184578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 413/839] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:53:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:56.190202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:56.214260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:56.222750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:56.223738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:56.224086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:56.224739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:56.225931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:56.226312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.227985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.248572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 427/840] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:56.249583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:56.303075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c9ae7-d1ed-4727-8293-3d9494b0add3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:56.339567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:56.340541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:56.340994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:56.341425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:56.353092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.354878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:56.374969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:56.376778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:56.378956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:56.378956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:56.378956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:56.378956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:56.378956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 414/841] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:56.383558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:56.384535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e45b6-4c0f-46e3-b3f1-ce52402f66d7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:56.392207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:56.421685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:56.422680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:56.423115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:56.423716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/70ebdb4e82064aad89b0dcec0a74cad2/roles/36d82dc24b2b44ee935e79f13d0f521c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:56.441498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=70ebdb4e82064aad89b0dcec0a74cad2, role_id=36d82dc24b2b44ee935e79f13d0f521c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.443292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.474902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:56.507168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f159074-f287-43b1-90ef-d5c9e5332faf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:56.541934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:56.542607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:56.542895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:56.543169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:56.543891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:56.544198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.545588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.557985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 415/842] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:56.563175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:56.658812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1aa16c-6189-453e-aa15-d0ce554998ad None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:56.691693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:56.692489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:56.692773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:56.693046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:56.700558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.702013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:53:56.716340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:56.717798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 24 09:53:56.718957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:56.721669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 416/843] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:53:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:56.723817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:56.794104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:56.794485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 428/844] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:53:56 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/70ebdb4e82064aad89b0dcec0a74cad2/roles/36d82dc24b2b44ee935e79f13d0f521c => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:53:56.834711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857becd-c60a-4a3c-a5f3-a43dc251a7cf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:56.874877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:56.875840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:56.876123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:56.876513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:56.878430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:56.878868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:56.880720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:56.901978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 417/845] 10.222.0.35 () {56 vars in 1135 bytes} [Tue May 24 09:53:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:56.908053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:57.024229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133fb44-7e6f-4c93-87e3-690d051d1158 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:57.061633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:57.062284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.062563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.062835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.090789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:57.135908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:57.137489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:57.252322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 429/846] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:53:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:53:57.258929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:57.311866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:57.363683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f4721-10e2-4226-b2e7-7e9a8cc4d821 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:57.399141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:57.399949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.400234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.400513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.411217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:57.412672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:57.431491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:57.432516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:57.433888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:57.433888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:57.433888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:57.433888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:57.433888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 418/847] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:57.438849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:57.502176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:57.548198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3b6c4-166a-4b29-b64f-55736a93858a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:57.584652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ead9dd-13f9-473d-86b4-a86c73e1ab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:57.585241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ead9dd-13f9-473d-86b4-a86c73e1ab95 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.585553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ead9dd-13f9-473d-86b4-a86c73e1ab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.585835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ead9dd-13f9-473d-86b4-a86c73e1ab95 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.586569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ead9dd-13f9-473d-86b4-a86c73e1ab95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:57.586872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ead9dd-13f9-473d-86b4-a86c73e1ab95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:57.588255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ead9dd-13f9-473d-86b4-a86c73e1ab95 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:57.601763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 430/848] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:57.607412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:57.650707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21310a-0529-41af-b54e-82fde3e59043 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.651064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21310a-0529-41af-b54e-82fde3e59043 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.651261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21310a-0529-41af-b54e-82fde3e59043 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.653413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 431/849] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:53:57.661534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a14ad-50ed-4c2c-a800-ad76610e5b59 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.661974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a14ad-50ed-4c2c-a800-ad76610e5b59 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.662531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a14ad-50ed-4c2c-a800-ad76610e5b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.683649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-693a14ad-50ed-4c2c-a800-ad76610e5b59 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:57.685493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-693a14ad-50ed-4c2c-a800-ad76610e5b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:57.720795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e6955-59fc-4bbe-aacd-db8f09d5f156 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:57.723125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:57.760746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:57.761859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.762322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.762763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.773371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 432/850] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:53:57.781308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:57.782813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:53:57.797780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:57.798747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:57.799770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 419/851] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:53:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:57.801549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a14ad-50ed-4c2c-a800-ad76610e5b59 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:57.815917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:57.866225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:57.934378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dc15cd-1863-4da0-9a40-6d52027c6447 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:57.936597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a14ad-50ed-4c2c-a800-ad76610e5b59 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:57.976528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba375af-683d-4b2c-8827-7b2cc5697e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:57.977579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba375af-683d-4b2c-8827-7b2cc5697e99 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.977763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fb029-79ba-495c-adec-e20b1514d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:57.978258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba375af-683d-4b2c-8827-7b2cc5697e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.978583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb029-79ba-495c-adec-e20b1514d757 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:57.978918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba375af-683d-4b2c-8827-7b2cc5697e99 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.979032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb029-79ba-495c-adec-e20b1514d757 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:57.979259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fb029-79ba-495c-adec-e20b1514d757 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:57.980122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fb029-79ba-495c-adec-e20b1514d757 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:57.980381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba375af-683d-4b2c-8827-7b2cc5697e99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:57.980873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba375af-683d-4b2c-8827-7b2cc5697e99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:57.981775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fb029-79ba-495c-adec-e20b1514d757 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:57.982696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba375af-683d-4b2c-8827-7b2cc5697e99 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:57.997246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 420/852] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:53:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:58.002768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb206-a59f-42fa-8b1d-a3d9617df748 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:58.003178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb206-a59f-42fa-8b1d-a3d9617df748 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:58.003685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb206-a59f-42fa-8b1d-a3d9617df748 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:58.005078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 421/853] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:58.007107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb206-a59f-42fa-8b1d-a3d9617df748 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:58.032027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:58.053317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 433/854] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:53:57 2022] POST /identity/v3/services => generated 251 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:53:58.053317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374fe4-272d-4b15-b116-7c81b020b553 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:58.053651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374fe4-272d-4b15-b116-7c81b020b553 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:58.054561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374fe4-272d-4b15-b116-7c81b020b553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:58.084651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-81374fe4-272d-4b15-b116-7c81b020b553 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:58.086419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-81374fe4-272d-4b15-b116-7c81b020b553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:58.146640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb206-a59f-42fa-8b1d-a3d9617df748 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:58.183127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 434/855] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:58 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:58.203142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:58.204004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:58.204292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:58.204576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:58.205583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=05f46d019e1348adabd023de481f0a3d) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:58.207265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374fe4-272d-4b15-b116-7c81b020b553 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:58.209444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:58.212556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:58.221097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 422/856] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:53:58 2022] GET /identity/v3/role_assignments?user.id=05f46d019e1348adabd023de481f0a3d&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:53:58.260747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:58.332183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374fe4-272d-4b15-b116-7c81b020b553 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:58.371659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:58.372912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:58.373440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:58.373943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:58.374861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:58.376489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:58.390944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:58.392028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:53:58.392906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:53:58.396823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 435/857] 10.222.0.35 () {56 vars in 1132 bytes} [Tue May 24 09:53:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:53:58.402661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:58.496808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:58.522195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b025da-fcbc-4f1d-bab7-ebed6cf3ff84 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:58.559800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd5f1e-8663-4b81-93ee-437978bdd0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:58.560480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd5f1e-8663-4b81-93ee-437978bdd0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:58.560767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd5f1e-8663-4b81-93ee-437978bdd0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:58.561046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd5f1e-8663-4b81-93ee-437978bdd0c7 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:58.561796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd5f1e-8663-4b81-93ee-437978bdd0c7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:58.562108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd5f1e-8663-4b81-93ee-437978bdd0c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:58.563483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd5f1e-8663-4b81-93ee-437978bdd0c7 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:58.569722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 423/858] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:53:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:58.575076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:58.679799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:58.685844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fffe7-e428-4e29-858c-53cd6fcd027b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:58.723327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428670c-ed9a-4d54-822b-224f77e631a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:58.724005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428670c-ed9a-4d54-822b-224f77e631a4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:58.724293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428670c-ed9a-4d54-822b-224f77e631a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:58.724574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428670c-ed9a-4d54-822b-224f77e631a4 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:58.725307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428670c-ed9a-4d54-822b-224f77e631a4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:58.725625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428670c-ed9a-4d54-822b-224f77e631a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:58.727013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428670c-ed9a-4d54-822b-224f77e631a4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:58.732265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 436/859] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:53:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:59.199163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8119fb-57f8-43d6-9270-efbed313c842 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:59.199452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8119fb-57f8-43d6-9270-efbed313c842 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:59.199737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8119fb-57f8-43d6-9270-efbed313c842 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:59.201387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 424/860] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:53:59.207268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780348c0-9949-430b-99b4-d092339d1022 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:59.207668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780348c0-9949-430b-99b4-d092339d1022 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:59.207956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780348c0-9949-430b-99b4-d092339d1022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:59.224922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-780348c0-9949-430b-99b4-d092339d1022 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:53:59.226145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-780348c0-9949-430b-99b4-d092339d1022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:53:59.298771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 437/861] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:53:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:53:59.322721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8119fb-57f8-43d6-9270-efbed313c842 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:59.434570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8119fb-57f8-43d6-9270-efbed313c842 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:59.473598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:59.474411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:59.474700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:59.474981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:59.486415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:53:59.495240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:59.496225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:59.497114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 425/862] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:53:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:59.502385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780348c0-9949-430b-99b4-d092339d1022 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:59.602038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:59.609033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780348c0-9949-430b-99b4-d092339d1022 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:59.645298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:59.645992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:59.646280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:59.646562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:59.647318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:59.647630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:59.649024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:59.657442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 438/863] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:53:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:53:59.663456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:59.673475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:53:59.771158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb0f4a-3b8c-4890-aad1-ca38f6765e05 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:59.806807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:59.807645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:59.807933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:59.808203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:59.821194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:59.822663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:53:59.840501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:53:59.841472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:53:59.842779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:53:59.842779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:53:59.842779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:53:59.842779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:53:59.842779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 426/864] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:53:59.848186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:53:59.957445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76a52f9-10a6-4778-8084-48306a44d743 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:53:59.993584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:53:59.994263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:53:59.994550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:53:59.994831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:53:59.995571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:53:59.995881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:53:59.997278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:00.010317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 439/865] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:00.015437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:00.085811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:00.129304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08de94e2-c9bf-458f-913f-6e4d3670cc01 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:00.169315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:00.169315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.169315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.169315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.178685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:00.180139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:00.195961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:00.196930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:00.197967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 427/866] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:54:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:00.204845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:00.225399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:00.320443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e339-3887-47a3-87f1-b8756a1ecb6f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:00.357045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4e1f8-f4fd-4655-8341-a93ffa4c61c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:00.358095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e1f8-f4fd-4655-8341-a93ffa4c61c0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.358595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e1f8-f4fd-4655-8341-a93ffa4c61c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.359082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4e1f8-f4fd-4655-8341-a93ffa4c61c0 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.360128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4e1f8-f4fd-4655-8341-a93ffa4c61c0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:00.360640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4e1f8-f4fd-4655-8341-a93ffa4c61c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:00.362855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4e1f8-f4fd-4655-8341-a93ffa4c61c0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:00.382079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 440/867] 10.222.0.35 () {56 vars in 1149 bytes} [Tue May 24 09:54:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:00.387966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:00.434460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b1e2b-baff-4cae-9bab-b021d60b5f00 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.434793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b1e2b-baff-4cae-9bab-b021d60b5f00 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.435023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b1e2b-baff-4cae-9bab-b021d60b5f00 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.436634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 441/868] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:00.441915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a41d-f9c3-408d-bf55-c11b86e4e09c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.442198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a41d-f9c3-408d-bf55-c11b86e4e09c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.442477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182a41d-f9c3-408d-bf55-c11b86e4e09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.461478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c182a41d-f9c3-408d-bf55-c11b86e4e09c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:00.462670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c182a41d-f9c3-408d-bf55-c11b86e4e09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:00.509467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771df09-0fc3-4b33-8f48-214701cb7fb1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:00.529932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:00.549568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 442/869] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:00.555425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d93fd-e47a-4db6-b817-d6d7a3c486b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:00.556086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d93fd-e47a-4db6-b817-d6d7a3c486b8 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.556372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d93fd-e47a-4db6-b817-d6d7a3c486b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.556646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d93fd-e47a-4db6-b817-d6d7a3c486b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.572340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a41d-f9c3-408d-bf55-c11b86e4e09c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:00.575733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:00.636497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d93fd-e47a-4db6-b817-d6d7a3c486b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:00.638266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d93fd-e47a-4db6-b817-d6d7a3c486b8 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:00.697446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182a41d-f9c3-408d-bf55-c11b86e4e09c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:00.734887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092dbc4-f800-4543-b1f5-6fea40375dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:00.735677 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092dbc4-f800-4543-b1f5-6fea40375dda None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.735959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092dbc4-f800-4543-b1f5-6fea40375dda None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.736232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092dbc4-f800-4543-b1f5-6fea40375dda None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.736884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092dbc4-f800-4543-b1f5-6fea40375dda None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:00.738890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092dbc4-f800-4543-b1f5-6fea40375dda None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c092dbc4-f800-4543-b1f5-6fea40375dda None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:00.753366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:00.754533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:00.755411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 443/870] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:54:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:00.756770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a80c-c804-44d1-8174-da5d0d9c6c85 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.756770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a80c-c804-44d1-8174-da5d0d9c6c85 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.756770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925a80c-c804-44d1-8174-da5d0d9c6c85 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.758022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 444/871] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:00.762291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a80c-c804-44d1-8174-da5d0d9c6c85 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:00.770285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 428/872] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:00 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:00.770285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3514fda-ecc5-403d-8f3a-fbd3e305fefb None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.770285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3514fda-ecc5-403d-8f3a-fbd3e305fefb None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.770285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3514fda-ecc5-403d-8f3a-fbd3e305fefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.786751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d3514fda-ecc5-403d-8f3a-fbd3e305fefb None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:00.787937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d3514fda-ecc5-403d-8f3a-fbd3e305fefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:00.803550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:00.839939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:00.859014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 429/873] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:00 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:00.860178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3514fda-ecc5-403d-8f3a-fbd3e305fefb None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:00.884199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925a80c-c804-44d1-8174-da5d0d9c6c85 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:00.925892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:00.926939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:00.927454 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:00.927963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:00.928806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:00.929118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:00.930532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:00.936007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 445/874] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:54:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:00.936818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:00.976550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3514fda-ecc5-403d-8f3a-fbd3e305fefb None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.013918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.014717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.014999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.015275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.026868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.028320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.037419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:01.047284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:01.048264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:01.049152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:01.049152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:01.049152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:01.049152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:01.049424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 430/875] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:01.050787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.060627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7426d7-a1a0-4f70-a159-964cc9266ea9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.091686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.096940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.097777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.098066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.098348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.099011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.100393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:01.112820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:01.113920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:01.114806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 446/876] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:01.115637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.160122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b646ec4-ba08-4e47-bfd3-63b9c491599d None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.195575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5d634-357f-40fe-9586-5214bf785cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.196329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5d634-357f-40fe-9586-5214bf785cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.196607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5d634-357f-40fe-9586-5214bf785cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.196910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5d634-357f-40fe-9586-5214bf785cf9 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.197639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5d634-357f-40fe-9586-5214bf785cf9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:01.197969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5d634-357f-40fe-9586-5214bf785cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.199391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5d634-357f-40fe-9586-5214bf785cf9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.204231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 431/877] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:01.211744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74300e4-f48b-4cb0-b8fc-0830260562d7 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.211744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74300e4-f48b-4cb0-b8fc-0830260562d7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.211744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74300e4-f48b-4cb0-b8fc-0830260562d7 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.211744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 432/878] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:01.211744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74300e4-f48b-4cb0-b8fc-0830260562d7 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.224999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4405500a-611c-464d-ab7e-d16c49086060 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.258377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8faaf-895c-4da4-b93f-73dee2e95bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.259032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8faaf-895c-4da4-b93f-73dee2e95bde None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.259314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8faaf-895c-4da4-b93f-73dee2e95bde None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.259591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8faaf-895c-4da4-b93f-73dee2e95bde None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.260317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8faaf-895c-4da4-b93f-73dee2e95bde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:01.260621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8faaf-895c-4da4-b93f-73dee2e95bde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.262019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f8faaf-895c-4da4-b93f-73dee2e95bde None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.272554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.274829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 447/879] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:01.276071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ee7b0-c194-49b9-89f0-95f218179c24 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.276357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ee7b0-c194-49b9-89f0-95f218179c24 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.277066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ee7b0-c194-49b9-89f0-95f218179c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.292842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-572ee7b0-c194-49b9-89f0-95f218179c24 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:01.294064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-572ee7b0-c194-49b9-89f0-95f218179c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:01.311202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74300e4-f48b-4cb0-b8fc-0830260562d7 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.343135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.344155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d2074-2180-404a-aa1e-348131033564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.344803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d2074-2180-404a-aa1e-348131033564 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.345086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d2074-2180-404a-aa1e-348131033564 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.345500 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d2074-2180-404a-aa1e-348131033564 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.346219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d2074-2180-404a-aa1e-348131033564 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:01.346526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d2074-2180-404a-aa1e-348131033564 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.348498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d2074-2180-404a-aa1e-348131033564 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.352956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 433/880] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:01.354522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d2074-2180-404a-aa1e-348131033564 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.360260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 448/881] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:01.361083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ee7b0-c194-49b9-89f0-95f218179c24 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.454119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d2074-2180-404a-aa1e-348131033564 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.462818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ee7b0-c194-49b9-89f0-95f218179c24 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.480001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.488477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.505003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.505003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.505003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.505003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] RBAC: Adding query filter params (interface=public, service_id=1439529ae5c6465abbde1273e2601a4a, region_id=RegionOne) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:01.505003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.505003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.505679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.505792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 434/882] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/endpoints?service_id=1439529ae5c6465abbde1273e2601a4a&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:01.505879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.505970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.506053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.506053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.506053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.506268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:01.520267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:01.521374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 449/883] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:01.522326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.570186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.601713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6837f-d330-4b5c-b8aa-347524c7abec None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.622525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683272f-a5bd-42be-8191-66bc2b11f6b2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.635163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.635828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.636117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.636389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.637094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:01.637429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.638796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.643541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 435/884] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:01.644453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.657602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.658292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.658572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.658845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.659817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.661244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.709847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.758021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54283e0-794f-4fc7-9a62-3366e5c026d6 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.771140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 450/885] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:54:01 2022] POST /identity/v3/services => generated 269 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:01.778442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:01.816713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.818016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.818016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.818314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.818975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.820712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.828400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 436/886] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/services => generated 1268 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 09:54:01.910597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c90fb-5783-4fc8-bebc-69fbfabd4bde None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:01.921998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:01.951609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab173d31-31d0-4c26-8231-31777e3fda47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:01.952471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173d31-31d0-4c26-8231-31777e3fda47 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:01.952857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173d31-31d0-4c26-8231-31777e3fda47 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:01.953235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab173d31-31d0-4c26-8231-31777e3fda47 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:01.954209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab173d31-31d0-4c26-8231-31777e3fda47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:01.954624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab173d31-31d0-4c26-8231-31777e3fda47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:01.956756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab173d31-31d0-4c26-8231-31777e3fda47 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:01.972228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 451/887] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:01.978342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:02.098837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee475b96-36c8-4156-8cb1-90ebce346c14 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:02.117703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:02.135756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139a730-6589-420d-827f-bfa386967d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:02.136501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139a730-6589-420d-827f-bfa386967d29 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:02.136854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139a730-6589-420d-827f-bfa386967d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:02.137196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139a730-6589-420d-827f-bfa386967d29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:02.138114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139a730-6589-420d-827f-bfa386967d29 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=70ebdb4e82064aad89b0dcec0a74cad2) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:02.138494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139a730-6589-420d-827f-bfa386967d29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:02.140498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d139a730-6589-420d-827f-bfa386967d29 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:02.150778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 437/888] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:01 2022] GET /identity/v3/role_assignments?user.id=70ebdb4e82064aad89b0dcec0a74cad2&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:03.949562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae062ce-c213-4675-87ab-ed39e6dd809a None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:03.950591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae062ce-c213-4675-87ab-ed39e6dd809a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:03.951110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae062ce-c213-4675-87ab-ed39e6dd809a None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:03.952931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 452/889] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:03.959527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8589f78-91ee-40d5-870d-9e50be6f5986 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:03.960056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8589f78-91ee-40d5-870d-9e50be6f5986 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:03.960454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8589f78-91ee-40d5-870d-9e50be6f5986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:03.981299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f8589f78-91ee-40d5-870d-9e50be6f5986 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:03.982524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f8589f78-91ee-40d5-870d-9e50be6f5986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:04.072067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 438/890] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:04.094395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae062ce-c213-4675-87ab-ed39e6dd809a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.212998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae062ce-c213-4675-87ab-ed39e6dd809a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:04.218949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a3cf0-5c6f-4b75-98ca-60d88787b489 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.219237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a3cf0-5c6f-4b75-98ca-60d88787b489 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.219516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a3cf0-5c6f-4b75-98ca-60d88787b489 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.221180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 439/891] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:04.227039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60b0b-eb11-4826-a85c-38291cce4c4a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.227328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60b0b-eb11-4826-a85c-38291cce4c4a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.227607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a60b0b-eb11-4826-a85c-38291cce4c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.245228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c2a60b0b-eb11-4826-a85c-38291cce4c4a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:04.246546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c2a60b0b-eb11-4826-a85c-38291cce4c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:04.252458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:04.254037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.254529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.255007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.255965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:04.257651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:04.315231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 440/892] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:04.317595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab1de-7e14-4b9d-b069-10907d6dc787 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.317892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab1de-7e14-4b9d-b069-10907d6dc787 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.318218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ab1de-7e14-4b9d-b069-10907d6dc787 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.319712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 441/893] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:04.325445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e20f0c-790b-44ca-a0af-94369f5b558c None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.325738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e20f0c-790b-44ca-a0af-94369f5b558c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.326024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e20f0c-790b-44ca-a0af-94369f5b558c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.335426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 453/894] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:54:04 2022] POST /identity/v3/services => generated 241 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:04.341261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.347156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-69e20f0c-790b-44ca-a0af-94369f5b558c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:04.348701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-69e20f0c-790b-44ca-a0af-94369f5b558c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:04.389665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:04.401497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:04.453636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 442/895] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:04.477553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da42f6-4cd9-474a-b55d-9506cbffb8b5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:04.477926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e20f0c-790b-44ca-a0af-94369f5b558c None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.514993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:04.515753 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.516033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.516306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.517167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:04.518756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:04.531776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:04.532816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:54:04.535159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:04.538826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 454/896] 10.222.0.35 () {56 vars in 1132 bytes} [Tue May 24 09:54:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:04.539716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.602938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e20f0c-790b-44ca-a0af-94369f5b558c None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:04.639739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f77e7f-eef9-49e4-adbf-ed8730608d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:04.640547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f77e7f-eef9-49e4-adbf-ed8730608d6b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.640834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f77e7f-eef9-49e4-adbf-ed8730608d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.641115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f77e7f-eef9-49e4-adbf-ed8730608d6b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.641821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f77e7f-eef9-49e4-adbf-ed8730608d6b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:04.643193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f77e7f-eef9-49e4-adbf-ed8730608d6b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:04.645628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:04.655292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca2fb0-6c8d-4cae-9318-33d9099c1fee None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-15f77e7f-eef9-49e4-adbf-ed8730608d6b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:04.662099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:04.663089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:04.663961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 443/897] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:54:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:04.664753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91250779-2a35-41f8-a4f3-a1c03bf8f882 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.664753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91250779-2a35-41f8-a4f3-a1c03bf8f882 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.664753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91250779-2a35-41f8-a4f3-a1c03bf8f882 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.664753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 444/898] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:04.665005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91250779-2a35-41f8-a4f3-a1c03bf8f882 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.697546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:04.702218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b552822-cf9f-4d27-ad0a-766123687990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:04.703067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b552822-cf9f-4d27-ad0a-766123687990 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.703515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b552822-cf9f-4d27-ad0a-766123687990 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.710095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b552822-cf9f-4d27-ad0a-766123687990 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.710095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b552822-cf9f-4d27-ad0a-766123687990 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:04.710095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b552822-cf9f-4d27-ad0a-766123687990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:04.710095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b552822-cf9f-4d27-ad0a-766123687990 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:04.712614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 455/899] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:04.713850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f18f96-0600-4855-926d-2efbc6b8392e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.714009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f18f96-0600-4855-926d-2efbc6b8392e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.714267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f18f96-0600-4855-926d-2efbc6b8392e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.732547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-69f18f96-0600-4855-926d-2efbc6b8392e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:04.733694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-69f18f96-0600-4855-926d-2efbc6b8392e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:04.775606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91250779-2a35-41f8-a4f3-a1c03bf8f882 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:04.806265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 456/900] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:04.813290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:04.813967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.814302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.814578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.815286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:04.815670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:04.817259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:04.818172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f18f96-0600-4855-926d-2efbc6b8392e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.827399 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 445/901] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:54:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:04.828365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.920477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:04.927318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:04.933854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1aee6-5fad-4195-b4c5-d0ab1fb482d2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:04.937860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f18f96-0600-4855-926d-2efbc6b8392e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:04.970064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:04.970764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.971068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.971326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.972724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:04.973218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:04.974741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:04.977680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:04.978834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:04.979290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:04.979737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:04.980138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 446/902] 10.222.0.35 () {56 vars in 1114 bytes} [Tue May 24 09:54:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:04.980829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:04.980944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:04.981529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:04.982944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:04.990122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 457/903] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:04.991001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:05.091545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef70d347-973d-412c-a14c-e41cdce5443c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:05.099534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb28b8e-f79d-4d72-98b6-151b2b568f4a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:05.130133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:05.130944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:05.131229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:05.131502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:05.134815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:05.135484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:05.135769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:05.136048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:05.136804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:05.138219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:05.146577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:05.154964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:05.156111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:05.157120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 447/904] 10.222.0.35 () {56 vars in 1120 bytes} [Tue May 24 09:54:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:05.157944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:05.178766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:05.261485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bac156-4f9b-43f6-b145-38abec3c81a8 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:05.274689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 458/905] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:54:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:05.276024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:05.301009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:05.307630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:05.308322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:05.308611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:05.308894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:05.309861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] RBAC: Adding query filter params (interface=public, service_id=bd5109b7e83f41219e54dc61112ef481, region_id=RegionOne) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:05.310185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:05.311698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:05.323609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 448/906] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:54:05 2022] GET /identity/v3/endpoints?service_id=bd5109b7e83f41219e54dc61112ef481&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:05.324636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:05.424555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:05.434757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09beb21-ef02-4c5b-b806-19a684afc80e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:05.450072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b35c8b0-bafb-4f11-a7ba-05f87c32016f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:05.483256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:05.483938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:05.484246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:05.484493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:05.485310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:05.485624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:05.487027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:05.498946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 459/907] 10.222.0.35 () {56 vars in 1115 bytes} [Tue May 24 09:54:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:05.500888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:05.502285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:05.502285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:05.502285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:05.503608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:05.505167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:05.505922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:05.513371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 449/908] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:54:05 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 09:54:05.597913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:05.631421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d9c70-b202-49f7-ae63-e1b3245ce2aa None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:05.670021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:05.670832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:05.671118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:05.671393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:05.684208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:05.685730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:05.704587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:05.705581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:05.707194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:05.707194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:05.707194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:05.707194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:05.707194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 460/909] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:05.714550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:05.820748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f67543-ba90-4242-ba86-72b2cc521fbb None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:05.855724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:05.856392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:05.856674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:05.856951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:05.857707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:05.858015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:05.859374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:05.871691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 450/910] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:05.877101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:05.984996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c730dabe-d4d6-464c-aa52-e17ef8988f69 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:06.009813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:06.022153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:06.022975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.023258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.023534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.031479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:06.032942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:06.048663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:06.049792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:06.051469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 461/911] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:54:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:06.058387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:06.062059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:06.168799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08693dfd-68ed-49c0-812d-dcc47d9c9626 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:06.205479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:06.206189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.206513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.206828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.207879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:06.208187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:06.209635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:06.228414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 451/912] 10.222.0.35 () {56 vars in 1149 bytes} [Tue May 24 09:54:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:06.233998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:06.341545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d97ce1-d8df-4cf7-bc20-3e12a44a2408 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:06.376765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e7706d-77ac-4cba-9773-a2c524b94031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:06.377848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7706d-77ac-4cba-9773-a2c524b94031 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.378881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7706d-77ac-4cba-9773-a2c524b94031 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.379361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e7706d-77ac-4cba-9773-a2c524b94031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.428799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:06.454514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7706d-77ac-4cba-9773-a2c524b94031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:06.456538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e7706d-77ac-4cba-9773-a2c524b94031 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:06.562395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 462/913] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:06 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:06.573078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:06.623050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f634-ab1e-4bd4-921e-4a86f394bcee None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.623366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f634-ab1e-4bd4-921e-4a86f394bcee None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.623608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f634-ab1e-4bd4-921e-4a86f394bcee None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.624959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 463/914] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:06.630835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd53986-4960-4b19-9077-21f3dbed2783 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.631303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd53986-4960-4b19-9077-21f3dbed2783 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.631670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd53986-4960-4b19-9077-21f3dbed2783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.636647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:06.652585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1cd53986-4960-4b19-9077-21f3dbed2783 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:06.654329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1cd53986-4960-4b19-9077-21f3dbed2783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:06.685190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc0f4d5-75c1-4ca9-97f0-31f4f4d46326 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:06.725617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:06.726828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.727587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.728501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.742480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:06.744128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:06.745608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 464/915] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:06.762355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd53986-4960-4b19-9077-21f3dbed2783 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:06.764144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:06.765253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:06.766450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:06.766450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:06.766450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:06.766450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:06.774678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 452/916] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:06.775557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:06.779657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:06.895533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd53986-4960-4b19-9077-21f3dbed2783 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:06.900458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf4155-b471-46e8-9817-80b370293f9f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:06.918422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:06.938949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:06.940180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.941149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.941702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.942620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:06.942834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:06.943605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:06.944005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:06.944382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:06.944827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:06.945366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:06.945814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:06.947258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:06.963920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 453/917] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:06.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:06.965300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:06.966392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 465/918] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:06.968683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:06.969503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:07.051394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:07.083373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0666c7b-8063-426b-80cf-0bc0e92c059b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.114662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65200cbf-47db-4d0b-b22c-3471c060077a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.120917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.121625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.121915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.122195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.122973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:07.123280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.124708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.136282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 466/919] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:07.137574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:07.158123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.159415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.159916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.160403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.169702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.171282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:07.192172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:07.193336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:07.194479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:07.201184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 454/920] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:07.202172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:07.244036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:07.273399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff0600-f4b5-47d7-9eee-0264299ab0f4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.322757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.323603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.323990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.324371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.324736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e6815b-7cd7-4d05-9c51-affb3b3a6ce1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.325337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:07.325750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.328375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.330693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:07.334445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 467/921] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:07.340341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:07.373437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.374725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.375377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.375881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.377560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:07.378133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.379941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.400927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 455/922] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:07.401691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:07.465913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad20a25-6e5d-4ff4-bad9-8fa9ccd866d9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.514919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:07.516818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.517995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.519602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.520372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.521566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] RBAC: Adding query filter params (interface=public, service_id=18689cecc2b148e7959936b74a8fafa2, region_id=RegionOne) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:07.522406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.524261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.524678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88abdb-cb13-4c1f-98a0-4c152cfad4c3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.534297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 468/923] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:54:07 2022] GET /identity/v3/endpoints?service_id=18689cecc2b148e7959936b74a8fafa2&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:07.540760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:07.565327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1ef6c-0e84-4ab0-ae4d-d7f6b4d7c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.566595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1ef6c-0e84-4ab0-ae4d-d7f6b4d7c283 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.566879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1ef6c-0e84-4ab0-ae4d-d7f6b4d7c283 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.567156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1ef6c-0e84-4ab0-ae4d-d7f6b4d7c283 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.568104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1ef6c-0e84-4ab0-ae4d-d7f6b4d7c283 None admin] RBAC: Adding query filter params (role.id=e46f6b11f37648bba7c2f9d6e5bf7091, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=70ebdb4e82064aad89b0dcec0a74cad2) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:07.568420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1ef6c-0e84-4ab0-ae4d-d7f6b4d7c283 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.570004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1ef6c-0e84-4ab0-ae4d-d7f6b4d7c283 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.585470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 456/924] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:07 2022] GET /identity/v3/role_assignments?user.id=70ebdb4e82064aad89b0dcec0a74cad2&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e46f6b11f37648bba7c2f9d6e5bf7091 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:07.623118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:07.662881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011245d-e74b-4b06-a717-62dfb2965c6d None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.663323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011245d-e74b-4b06-a717-62dfb2965c6d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.663711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011245d-e74b-4b06-a717-62dfb2965c6d None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.666858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b0863e-b409-4ce4-8d5f-b2712fe7a89e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.668965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 457/925] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:07.671159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4918523-cbc8-43c7-a89a-66a2bdd042a9 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.671459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4918523-cbc8-43c7-a89a-66a2bdd042a9 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.671745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4918523-cbc8-43c7-a89a-66a2bdd042a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.690040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c4918523-cbc8-43c7-a89a-66a2bdd042a9 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:07.691932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c4918523-cbc8-43c7-a89a-66a2bdd042a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:07.712059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.712975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.713287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.713572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.714714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.716358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.726814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 469/926] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:54:07 2022] GET /identity/v3/services => generated 1497 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 09:54:07.779515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 458/927] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:07.803851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:07.915390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f36a3-1dfc-46ab-8cfb-709dd7605904 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.915677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f36a3-1dfc-46ab-8cfb-709dd7605904 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.915958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f36a3-1dfc-46ab-8cfb-709dd7605904 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.917549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 459/928] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:07.921676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6027d-1f06-4936-8875-27e66cd91667 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:07.928607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a50f-1e03-44cc-8217-69d4a1029b6f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.929016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a50f-1e03-44cc-8217-69d4a1029b6f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.929437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564a50f-1e03-44cc-8217-69d4a1029b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.951899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2564a50f-1e03-44cc-8217-69d4a1029b6f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:07.953557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2564a50f-1e03-44cc-8217-69d4a1029b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:07.959162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:07.971439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:07.973363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:07.973864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:07.974382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:07.975089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:07.976557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:07.977989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING keystone.server.flask.common [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 24 09:54:08.035209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.sql.core [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 09:54:08.035209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 09:54:08.035209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [parameters: {'id': 'f885c7290b1049a1bbc6a868122db848', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 09:54:08.035209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80405) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 09:54:08.040744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 460/929] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f885c7290b1049a1bbc6a868122db848.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f885c7290b1049a1bbc6a868122db848. May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result = self._query(query) May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application conn.query(q) May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 09:54:08.054210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result.read() May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 24 09:54:08.055526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return project_ref.to_dict() May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application next(self.gen) May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application yield resource May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application next(self.gen) May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.session.rollback() May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.force_reraise() May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise self.value May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application session.commit() May 24 09:54:08.056505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._prepare_impl() May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.session.flush() May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._flush(objects) May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application compat.raise_( May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application flush_context.execute() May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 09:54:08.057496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rec.execute(self) May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application c = connection._execute_20( May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 09:54:08.058426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result = self._query(query) May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application conn.query(q) May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result.read() May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 09:54:08.059240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [parameters: {'id': 'f885c7290b1049a1bbc6a868122db848', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:08.060150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:08.060950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.Conflict( May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f885c7290b1049a1bbc6a868122db848. May 24 09:54:08.062058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.064615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 470/930] 10.222.0.35 () {60 vars in 1140 bytes} [Tue May 24 09:54:07 2022] POST /identity/v3/projects => generated 262 bytes in 261 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:08.069338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564a50f-1e03-44cc-8217-69d4a1029b6f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:08.072878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:08.087270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:08.195999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564a50f-1e03-44cc-8217-69d4a1029b6f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:08.213455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487b135-bc2f-49e6-b03e-bd2bef0133f8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:08.237006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:08.238417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:08.238946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:08.239403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:08.240244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:08.242040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:08.250147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:08.251080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:08.251431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:08.251775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:08.257697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:08.258754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 24 09:54:08.259618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:08.260422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:08.262596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:08.262965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 461/931] 10.222.0.35 () {56 vars in 1132 bytes} [Tue May 24 09:54:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:08.269707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:08.284840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:08.285910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:08.286841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:08.291049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 471/932] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:08.292020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:08.343250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:08.380654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:08.402776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598918af-c6f7-4ada-92c8-9f4aca32c218 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:08.405594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2516bb-f347-49b6-88d8-1acb547e25ea None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:08.445709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:08.445918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:08.446495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:08.446773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:08.447049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:08.447526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:08.447758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:08.448058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:08.448440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:08.448911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:08.449442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:08.450195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:08.450740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:08.452490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:08.460379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 462/933] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:08.461939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:08.486220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 472/934] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:54:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:08.586177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d249b-f076-4e38-88ca-11fb00e16860 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:08.624450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ad3e9d-4fcd-4026-83c3-c4d755f6e7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:08.625139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad3e9d-4fcd-4026-83c3-c4d755f6e7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:08.625461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad3e9d-4fcd-4026-83c3-c4d755f6e7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:08.625741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ad3e9d-4fcd-4026-83c3-c4d755f6e7a5 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:08.626612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad3e9d-4fcd-4026-83c3-c4d755f6e7a5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:08.626930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad3e9d-4fcd-4026-83c3-c4d755f6e7a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:08.628468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ad3e9d-4fcd-4026-83c3-c4d755f6e7a5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:08.634679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 463/935] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:08.640325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:08.751678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843430b-ca78-44ac-b468-35c342d7010d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:08.756613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:08.790878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b20f9-1dc0-4207-b4b3-ce0ff5e3b6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:08.791552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b20f9-1dc0-4207-b4b3-ce0ff5e3b6f9 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:08.791902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b20f9-1dc0-4207-b4b3-ce0ff5e3b6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:08.792480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b20f9-1dc0-4207-b4b3-ce0ff5e3b6f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:08.793240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b20f9-1dc0-4207-b4b3-ce0ff5e3b6f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:08.794662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b20f9-1dc0-4207-b4b3-ce0ff5e3b6f9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:08.868388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 473/936] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:54:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:09.876768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90357-f533-4898-8537-b2586cad7ec5 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:09.877320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90357-f533-4898-8537-b2586cad7ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:09.877320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f90357-f533-4898-8537-b2586cad7ec5 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:09.879622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 464/937] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:09.885008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ef471-b9f1-4078-a9fa-02512ca9a8ee None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:09.885458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ef471-b9f1-4078-a9fa-02512ca9a8ee None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:09.885645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ef471-b9f1-4078-a9fa-02512ca9a8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:09.902729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0a4ef471-b9f1-4078-a9fa-02512ca9a8ee None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:09.904225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0a4ef471-b9f1-4078-a9fa-02512ca9a8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:09.983170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 474/938] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:10.006242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90357-f533-4898-8537-b2586cad7ec5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.017463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:10.117436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f90357-f533-4898-8537-b2586cad7ec5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.149127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5f01b-6bb6-462f-a93f-1cff2a9f4393 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.149427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5f01b-6bb6-462f-a93f-1cff2a9f4393 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.149671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a5f01b-6bb6-462f-a93f-1cff2a9f4393 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.151199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 475/939] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:10.154345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.155162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.155447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.155725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.156367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d866b4e-1436-4400-ac21-0783ff4cc69d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.156666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d866b4e-1436-4400-ac21-0783ff4cc69d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.156949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d866b4e-1436-4400-ac21-0783ff4cc69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.167522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:10.174418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2d866b4e-1436-4400-ac21-0783ff4cc69d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:10.176632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:10.177595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:10.178452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:10.179231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2d866b4e-1436-4400-ac21-0783ff4cc69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:10.179304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 465/940] 10.222.0.35 () {56 vars in 1120 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:10.183712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.261037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:10.263176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 476/941] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:10.292857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d866b4e-1436-4400-ac21-0783ff4cc69d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.310138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841c7042-4b39-4cb6-990d-bbb8ef7aa036 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.313716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:10.348890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.352029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.352029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.352029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.352029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:10.352029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:10.352616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:10.361389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 466/942] 10.222.0.35 () {56 vars in 1115 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:10.369979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.417042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d866b4e-1436-4400-ac21-0783ff4cc69d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.459310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.460127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.460411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.460686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.461377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:10.462767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:10.473626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4848f57-e29a-468b-b331-082f18876390 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:10.476481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:10.477481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:10.478318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 477/943] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:10.483963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.505621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:10.509907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.510721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.511013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.511296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.522624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:10.524084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:10.547935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:10.548997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:10.550673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:10.550673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:10.550673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:10.550673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:10.552487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 467/944] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:10.558089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.595044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:10.617375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe48c7c-60ab-4aae-a6f1-5ed4a175a70f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.652605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.653473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.653855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.654235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.655055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:10.655491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:10.656962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:10.663982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 478/945] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:10.669287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.698859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1102627-0d60-4e68-bc82-4de1f06e6ea6 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.748658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.749361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.749655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.749930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.750955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:10.751877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:10.754194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:10.773899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 468/946] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:10.774822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.802902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710850-63c4-4c8d-bdfa-9f8ab7dda014 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.821257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:10.850518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.851016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.851305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.851578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.852293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:10.852661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:10.854134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:10.859966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 479/947] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:10.869620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.882557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:10.887382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c38c25-a054-40a3-8871-fcc57043e201 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:10.924389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878ca08-4f45-4d27-b9ef-1c7855f4a3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:10.926007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ca08-4f45-4d27-b9ef-1c7855f4a3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.926295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ca08-4f45-4d27-b9ef-1c7855f4a3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.926579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878ca08-4f45-4d27-b9ef-1c7855f4a3e4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.934372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878ca08-4f45-4d27-b9ef-1c7855f4a3e4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:10.935992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878ca08-4f45-4d27-b9ef-1c7855f4a3e4 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6878ca08-4f45-4d27-b9ef-1c7855f4a3e4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:10.951851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:10.959879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 469/948] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd527a-095e-4262-8289-f9a81a513a74 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:10.961026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd527a-095e-4262-8289-f9a81a513a74 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:10.961958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd527a-095e-4262-8289-f9a81a513a74 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:10.961958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 470/949] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:10.961958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd527a-095e-4262-8289-f9a81a513a74 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:10.990634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1e20b-3622-4926-bc67-f58b484380c2 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.030380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc34d1-768c-43d4-9192-7e3885013ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.031058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc34d1-768c-43d4-9192-7e3885013ad4 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.031332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc34d1-768c-43d4-9192-7e3885013ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.031607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc34d1-768c-43d4-9192-7e3885013ad4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.032375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc34d1-768c-43d4-9192-7e3885013ad4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.033831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc34d1-768c-43d4-9192-7e3885013ad4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.079203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd527a-095e-4262-8289-f9a81a513a74 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.080863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:11.114179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.114845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.115127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.115404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.116234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:11.116540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.118053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.125450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 480/950] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:54:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:54:11.127060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d949e-404b-4ed8-b0d8-c2471e87f742 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.127359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d949e-404b-4ed8-b0d8-c2471e87f742 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.127701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d949e-404b-4ed8-b0d8-c2471e87f742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.147100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 471/951] 10.222.0.35 () {56 vars in 1149 bytes} [Tue May 24 09:54:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:11.150487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-464d949e-404b-4ed8-b0d8-c2471e87f742 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:11.151779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-464d949e-404b-4ed8-b0d8-c2471e87f742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:11.152426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.227470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:11.248233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 481/952] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:11.248233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd761-654f-4123-9ed1-0772d18e4417 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.248233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd761-654f-4123-9ed1-0772d18e4417 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.248233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfd761-654f-4123-9ed1-0772d18e4417 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.248233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 482/953] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:11.252804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2104-710b-450a-80e0-db65c4fd3957 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.253655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2104-710b-450a-80e0-db65c4fd3957 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.253929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca2104-710b-450a-80e0-db65c4fd3957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.276767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-59ca2104-710b-450a-80e0-db65c4fd3957 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:11.278722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-59ca2104-710b-450a-80e0-db65c4fd3957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:11.299435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8325e-22fc-4cb7-9644-7947295b4d0c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.341942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.343262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.343262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.343695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.356994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.358417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 483/954] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:11.359599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2104-710b-450a-80e0-db65c4fd3957 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.359710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:11.380958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:11.382149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:11.383146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:11.383146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:11.383146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:11.383146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:11.383146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 472/955] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:11.383432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.401143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:11.475870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca2104-710b-450a-80e0-db65c4fd3957 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.492373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d877eba-585f-497e-a6e4-6db29c0d16a0 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.497819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:11.512202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.512878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.513162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.513465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.514190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.515636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.530114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.530905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.531187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.531462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.532118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.533511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:11.545370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:11.546529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:11.553604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 473/956] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:11.554596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.571870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.sql.core [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 09:54:11.571870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 09:54:11.571870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [parameters: {'user_id': '391337d6036542a88e6ab1c8c526bb92', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 09:54:11.571870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80405) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result = self._query(query) May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application conn.query(q) May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result.read() May 24 09:54:11.580787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 09:54:11.581791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application next(self.gen) May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application yield resource May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application next(self.gen) May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.session.rollback() May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.force_reraise() May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise self.value May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application session.commit() May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 24 09:54:11.583503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._prepare_impl() May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.session.flush() May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._flush(objects) May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application compat.raise_( May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application flush_context.execute() May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rec.execute(self) May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 24 09:54:11.585593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application _emit_insert_statements( May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result = connection._execute_20( May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ret = self._execute_context( May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 24 09:54:11.587954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.dialect.do_execute( May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result = self._query(query) May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application conn.query(q) May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application result.read() May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 24 09:54:11.591158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application packet.raise_for_error() May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [parameters: {'user_id': '391337d6036542a88e6ab1c8c526bb92', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:11.593683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 24 09:54:11.596101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 484/957] 10.222.0.35 () {60 vars in 1131 bytes} [Tue May 24 09:54:11 2022] POST /identity/v3/users => generated 160 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:11.599570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.656549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:11.690328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f0c8b5-9a66-4348-b4e7-edef836ece14 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.724009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96473fd8-3e70-4ab2-9d4d-ba088faee072 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.729914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.730583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.730862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.731420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.731962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:11.732263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.733813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.747060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 474/958] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:11.747800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.770022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.770690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.770972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.771259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.771962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:11.772265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.773879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.781727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 485/959] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:11.782551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.833296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:11.864971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae31a18d-f24b-465e-bb45-5a87f4a177ed None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.897390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae361ee-2ce6-4f13-b1ba-4975f517f0fb None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:11.901413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.902213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.902496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.902773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.908113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:11.914017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.915533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:11.931339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:11.932337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 24 09:54:11.933336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:11.936351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:11.937250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:11.937505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 475/960] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:11.937595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:11.937767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:11.938192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:11.946304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:11.947843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:11.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:11.964629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 486/961] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:11.970941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:12.053980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc371d-4b61-4a78-bc23-5551ce30c171 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:12.076261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732a781-934a-46b2-94e5-f80fd8886712 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:12.084869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:12.092728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:12.096290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:12.096290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:12.096290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:12.096290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:12.096290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:12.096841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:12.101800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 476/962] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:12.102538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:12.121540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:12.122400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:12.122785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:12.123146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:12.124105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:12.124518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:12.126530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:12.177305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 487/963] 10.222.0.35 () {56 vars in 1101 bytes} [Tue May 24 09:54:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:12.202736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:12.251912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b41fc-8155-4146-8c90-837caf74ba2c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:12.297046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e411c3ea-761b-4056-81e6-690dfafea0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:12.298123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411c3ea-761b-4056-81e6-690dfafea0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:12.298563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411c3ea-761b-4056-81e6-690dfafea0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:12.298985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411c3ea-761b-4056-81e6-690dfafea0d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:12.300823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411c3ea-761b-4056-81e6-690dfafea0d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:12.301324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411c3ea-761b-4056-81e6-690dfafea0d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:12.302999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e411c3ea-761b-4056-81e6-690dfafea0d1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:12.319772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 477/964] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:12.325981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:12.445751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05896b4-46a2-4f79-80fa-3a3bf81340a7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:12.483119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b0728a-4adc-4783-9626-67f6f4ecee5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:12.484024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0728a-4adc-4783-9626-67f6f4ecee5e None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:12.484304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0728a-4adc-4783-9626-67f6f4ecee5e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:12.484579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b0728a-4adc-4783-9626-67f6f4ecee5e None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/70ebdb4e82064aad89b0dcec0a74cad2/roles/e46f6b11f37648bba7c2f9d6e5bf7091` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:12.488066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0728a-4adc-4783-9626-67f6f4ecee5e None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=70ebdb4e82064aad89b0dcec0a74cad2, role_id=e46f6b11f37648bba7c2f9d6e5bf7091)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:12.492876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b0728a-4adc-4783-9626-67f6f4ecee5e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:12.560133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 488/965] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:54:12 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/70ebdb4e82064aad89b0dcec0a74cad2/roles/e46f6b11f37648bba7c2f9d6e5bf7091 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:54:13.439411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30937c2-182d-41bb-b888-b0d61ca94e96 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:13.439913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30937c2-182d-41bb-b888-b0d61ca94e96 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:13.439993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30937c2-182d-41bb-b888-b0d61ca94e96 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:13.442040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 478/966] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:13.447396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016f9a9-9f4f-492a-a4e2-bd8dabfdd83f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:13.447810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016f9a9-9f4f-492a-a4e2-bd8dabfdd83f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:13.447972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016f9a9-9f4f-492a-a4e2-bd8dabfdd83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:13.464788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b016f9a9-9f4f-492a-a4e2-bd8dabfdd83f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:13.466408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b016f9a9-9f4f-492a-a4e2-bd8dabfdd83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:13.484898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:13.533633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 489/967] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:13.557109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30937c2-182d-41bb-b888-b0d61ca94e96 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:13.660257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30937c2-182d-41bb-b888-b0d61ca94e96 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:13.683049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:13.697848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:13.698585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:13.698864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:13.699137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:13.699850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:13.701338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:13.715018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:13.716050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:13.716929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:13.721231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 479/968] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:13.723230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016f9a9-9f4f-492a-a4e2-bd8dabfdd83f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:13.828784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016f9a9-9f4f-492a-a4e2-bd8dabfdd83f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:13.862997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:13.863667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:13.863952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:13.864231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:13.864946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:13.865253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:13.866665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:13.876236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 490/969] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:13.877857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:13.981928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a735ef91-260a-4faa-afc0-251bf9411dcf None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:14.015358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:14.016026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.016309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.016585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.017312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:14.017625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:14.019416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:14.024409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 480/970] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:14.030206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:14.050831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:14.133021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666244fe-9c6a-4052-89a7-342b5baf5bd4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:14.177534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be5fa6-4f20-45b0-a3b5-fe382e2f0496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:14.178225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be5fa6-4f20-45b0-a3b5-fe382e2f0496 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.178508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be5fa6-4f20-45b0-a3b5-fe382e2f0496 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.178783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be5fa6-4f20-45b0-a3b5-fe382e2f0496 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.179618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be5fa6-4f20-45b0-a3b5-fe382e2f0496 None admin] RBAC: Adding query filter params (interface=admin, service_id=18689cecc2b148e7959936b74a8fafa2, region_id=RegionOne) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:14.179932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be5fa6-4f20-45b0-a3b5-fe382e2f0496 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:14.181372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3be5fa6-4f20-45b0-a3b5-fe382e2f0496 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:14.192113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 491/971] 10.222.0.35 () {56 vars in 1249 bytes} [Tue May 24 09:54:14 2022] GET /identity/v3/endpoints?service_id=18689cecc2b148e7959936b74a8fafa2&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:14.194065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:14.244679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:14.297316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef353e9-fcdd-4052-a492-f1be7697ef35 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:14.331646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1de364-4ce2-4e20-86b3-91cc126b6b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:14.332316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de364-4ce2-4e20-86b3-91cc126b6b85 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.332601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de364-4ce2-4e20-86b3-91cc126b6b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.332874 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1de364-4ce2-4e20-86b3-91cc126b6b85 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.333582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de364-4ce2-4e20-86b3-91cc126b6b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:14.334955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1de364-4ce2-4e20-86b3-91cc126b6b85 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:14.349613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 481/972] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:54:14 2022] GET /identity/v3/services => generated 1497 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 09:54:14.358041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254da13-a75d-4c46-9800-90cfe33d3051 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.358329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254da13-a75d-4c46-9800-90cfe33d3051 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.358609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254da13-a75d-4c46-9800-90cfe33d3051 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.360014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 492/973] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:14.366306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961f2a-5072-46df-b8ee-c1eb5840220c None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.366630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961f2a-5072-46df-b8ee-c1eb5840220c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.366875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53961f2a-5072-46df-b8ee-c1eb5840220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.384838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-53961f2a-5072-46df-b8ee-c1eb5840220c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:14.386090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-53961f2a-5072-46df-b8ee-c1eb5840220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:14.462600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 482/974] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:14.485261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254da13-a75d-4c46-9800-90cfe33d3051 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:14.579823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:14.599641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254da13-a75d-4c46-9800-90cfe33d3051 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:14.639623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff1b0d-1af6-4c4c-a639-2bb2788b0c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:14.640309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff1b0d-1af6-4c4c-a639-2bb2788b0c07 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.640590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff1b0d-1af6-4c4c-a639-2bb2788b0c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.640916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff1b0d-1af6-4c4c-a639-2bb2788b0c07 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.641702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff1b0d-1af6-4c4c-a639-2bb2788b0c07 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:14.643099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff1b0d-1af6-4c4c-a639-2bb2788b0c07 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:14.749302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 493/975] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:54:14 2022] POST /identity/v3/services => generated 242 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:14.811185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652436-49bf-4649-ba37-6a571088317d None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.811521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652436-49bf-4649-ba37-6a571088317d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.811757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09652436-49bf-4649-ba37-6a571088317d None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.813300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 483/976] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:14.819552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7c2ca-892c-4f4b-ab2e-0439141ed684 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:14.819877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7c2ca-892c-4f4b-ab2e-0439141ed684 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:14.820129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7c2ca-892c-4f4b-ab2e-0439141ed684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:14.838452 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6df7c2ca-892c-4f4b-ab2e-0439141ed684 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:14.839585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6df7c2ca-892c-4f4b-ab2e-0439141ed684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:14.925519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 494/977] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:14.949892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652436-49bf-4649-ba37-6a571088317d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:14.997723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:15.064907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09652436-49bf-4649-ba37-6a571088317d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:15.101165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:15.102011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:15.102372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:15.102737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:15.113512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:15.121930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:15.123061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:15.124104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:15.125574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 484/978] 10.222.0.35 () {56 vars in 1120 bytes} [Tue May 24 09:54:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:15.131717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7c2ca-892c-4f4b-ab2e-0439141ed684 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:15.208078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:15.243178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df7c2ca-892c-4f4b-ab2e-0439141ed684 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:15.278698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:15.279550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:15.280266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:15.280561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:15.281354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:15.281675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:15.283059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:15.296471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 495/979] 10.222.0.35 () {56 vars in 1115 bytes} [Tue May 24 09:54:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:15.298092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:15.402403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73253973-011e-4b65-a3d1-328ff7389681 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:15.403858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:15.436407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:15.437197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:15.437511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:15.437792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:15.448729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:15.450288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:15.468381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:15.469857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:15.470797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:15.470797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:15.470797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:15.470797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:15.473858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 485/980] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:15.476205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:15.579827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7843dc-2202-4fdd-b863-b5ab792a7b73 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:15.608296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:15.615223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:15.620716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:15.620716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:15.620716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:15.620716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:15.620716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:15.620716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:15.631087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 496/981] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:15.636456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:15.735704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf8ee8-9fde-44b3-97fc-73be605197ba None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:15.769675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:15.770510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:15.770751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:15.771021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:15.777903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:15.779342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:15.795412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:15.796390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 24 09:54:15.797408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:15.798582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 486/982] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:54:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:15.802464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:15.904729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06f4b8-b5d8-4100-9e12-20936d6f4c1c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:15.943294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:15.943979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:15.944259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:15.944582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:15.946330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:15.947000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:15.948744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:15.971813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 497/983] 10.222.0.35 () {56 vars in 1149 bytes} [Tue May 24 09:54:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:15.973892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:15.990638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:16.075566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759b25b-7629-48e7-a60a-fe717c669993 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:16.110269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:16.110917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.111197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.111471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.187029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:16.192621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:16.203647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:16.288546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 487/984] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:15 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:16.293933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:16.348388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:16.403921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b4cb8-8ae2-4b7e-9da5-80eaa091375c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:16.440030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a6c3c-4271-429b-bdc6-c69cdb271f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:16.441421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a6c3c-4271-429b-bdc6-c69cdb271f39 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.441752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a6c3c-4271-429b-bdc6-c69cdb271f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.441972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a6c3c-4271-429b-bdc6-c69cdb271f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.452318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a6c3c-4271-429b-bdc6-c69cdb271f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:16.453798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a6c3c-4271-429b-bdc6-c69cdb271f39 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-315a6c3c-4271-429b-bdc6-c69cdb271f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:16.472265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:16.473237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:16.474468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:16.474468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:16.474468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:16.474468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:16.477346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 498/985] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:16.479252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:16.580059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0959a9-067f-4315-8c8b-8a35ff23acc1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:16.586188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d007f-d858-47b1-a9ba-25e7d86c13e0 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.586657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d007f-d858-47b1-a9ba-25e7d86c13e0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.586736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d007f-d858-47b1-a9ba-25e7d86c13e0 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.588841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 499/986] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:16.593861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9236b8d-7c0c-4f4b-b4c4-dbba0ee71096 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.594152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9236b8d-7c0c-4f4b-b4c4-dbba0ee71096 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.594433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9236b8d-7c0c-4f4b-b4c4-dbba0ee71096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.599071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:16.611750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a9236b8d-7c0c-4f4b-b4c4-dbba0ee71096 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:16.613736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a9236b8d-7c0c-4f4b-b4c4-dbba0ee71096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:16.620615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:16.621387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.621625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.621900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.622621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:16.622926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:16.624275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:16.638129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 488/987] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:16.643572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:16.686914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 500/988] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:16.710444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9236b8d-7c0c-4f4b-b4c4-dbba0ee71096 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:16.741946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:16.746420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f753d70-44c4-4445-ac89-43ace02bfe38 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:16.784042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:16.784862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.785141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.785440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.792524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:16.793998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:16.809092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:16.810267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:16.811523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 489/989] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:16.816889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:16.824533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9236b8d-7c0c-4f4b-b4c4-dbba0ee71096 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:16.853421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:16.862108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:16.868084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.868084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.868084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.868084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:16.868084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:16.879600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:16.880805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 24 09:54:16.881709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:16.884236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 501/990] 10.222.0.35 () {56 vars in 1126 bytes} [Tue May 24 09:54:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:16.890291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:16.924286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26ea1-f93d-47d6-aef5-35b9a59987fa None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:16.960020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:16.960684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:16.960965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:16.961237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:16.962055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:16.962359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:16.963770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:16.976319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 490/991] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:16.982503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:16.994374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:17.001804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73edb028-51ce-4ce0-9c32-c20d1d192fc0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.038461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eb9183-d981-4394-98ff-cca3c5401473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.039136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb9183-d981-4394-98ff-cca3c5401473 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.039419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb9183-d981-4394-98ff-cca3c5401473 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.039700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eb9183-d981-4394-98ff-cca3c5401473 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.040409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb9183-d981-4394-98ff-cca3c5401473 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:17.040714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb9183-d981-4394-98ff-cca3c5401473 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.042126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eb9183-d981-4394-98ff-cca3c5401473 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.052474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 502/992] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:17.052474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2cb6a-cdc4-4b12-b310-ab2a355af0f5 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.052474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2cb6a-cdc4-4b12-b310-ab2a355af0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.052474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2cb6a-cdc4-4b12-b310-ab2a355af0f5 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.052474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 503/993] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:17.054110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2cb6a-cdc4-4b12-b310-ab2a355af0f5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:17.086478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:17.093349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed138c-b110-4833-bba1-ed6fa0fb69a3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.128498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c79020-3762-49d3-8abf-408b2be3d603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.128955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c79020-3762-49d3-8abf-408b2be3d603 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.129234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c79020-3762-49d3-8abf-408b2be3d603 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.129546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c79020-3762-49d3-8abf-408b2be3d603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.130366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c79020-3762-49d3-8abf-408b2be3d603 None admin] RBAC: Adding query filter params (role.id=e46f6b11f37648bba7c2f9d6e5bf7091, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=70ebdb4e82064aad89b0dcec0a74cad2) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:17.130673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c79020-3762-49d3-8abf-408b2be3d603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.132089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c79020-3762-49d3-8abf-408b2be3d603 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.137811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 491/994] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:16 2022] GET /identity/v3/role_assignments?user.id=70ebdb4e82064aad89b0dcec0a74cad2&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e46f6b11f37648bba7c2f9d6e5bf7091 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:17.139031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5ec83-bf9d-441c-b472-7114ec265666 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.139326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5ec83-bf9d-441c-b472-7114ec265666 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.139586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5ec83-bf9d-441c-b472-7114ec265666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.157681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-eff5ec83-bf9d-441c-b472-7114ec265666 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:17.158935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-eff5ec83-bf9d-441c-b472-7114ec265666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:17.186866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2cb6a-cdc4-4b12-b310-ab2a355af0f5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.229281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.229988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.230275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.230549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.231274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:17.231579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.233197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.243581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 504/995] 10.222.0.35 () {56 vars in 1122 bytes} [Tue May 24 09:54:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:17.243801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 492/996] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:17.244771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:17.256671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5ec83-bf9d-441c-b472-7114ec265666 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:17.297478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:17.370314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:17.384253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5ec83-bf9d-441c-b472-7114ec265666 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.388921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527a0359-4f6c-4983-89ef-17cd04ecabee None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.419961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.420760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.421042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.421344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.422050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.423401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.425405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.426152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.426442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.426738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.427473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.428839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:17.435368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:17.441548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:17.442612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 493/997] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:17.443456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:17.513299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 505/998] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:54:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:17.547668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5112c-9bf2-497a-82de-e230a4f52b61 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.590515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.591190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.591467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.591737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.592447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:17.592752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.594164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.600450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 494/999] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:54:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:17.606645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:17.707275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:17.728063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b694b68-91b5-4c4f-ba59-0eda862898cf None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.764787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.765475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.765758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.766028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.766735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:17.767037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.768412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.774493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 506/1000] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:54:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:17.780526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:17.796296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:17.883661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38c26aa-c707-494f-adce-90778446ee1e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:17.919011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c61a80-2db1-4b80-891c-464a426f047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:17.919943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61a80-2db1-4b80-891c-464a426f047a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:17.920370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61a80-2db1-4b80-891c-464a426f047a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:17.920803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c61a80-2db1-4b80-891c-464a426f047a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:17.921771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c61a80-2db1-4b80-891c-464a426f047a None admin] RBAC: Adding query filter params (interface=public, service_id=e7332a8cd496423da42f5738eff0f1fa, region_id=RegionOne) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:17.922229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c61a80-2db1-4b80-891c-464a426f047a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:17.923783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c61a80-2db1-4b80-891c-464a426f047a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:17.929718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 495/1001] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:54:17 2022] GET /identity/v3/endpoints?service_id=e7332a8cd496423da42f5738eff0f1fa&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:17.935074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:18.040502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8adb6e-599e-4c46-99cb-2db6d1899e94 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:18.075462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f60d41-7e89-4b6a-b040-bb9888b63309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:18.076127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60d41-7e89-4b6a-b040-bb9888b63309 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:18.076410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60d41-7e89-4b6a-b040-bb9888b63309 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:18.076684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f60d41-7e89-4b6a-b040-bb9888b63309 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:18.077398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f60d41-7e89-4b6a-b040-bb9888b63309 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:18.078771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f60d41-7e89-4b6a-b040-bb9888b63309 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:18.085764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 507/1002] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:54:17 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 09:54:19.431798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e66265-2fd1-4690-a545-4d37decfba1a None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:19.432310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e66265-2fd1-4690-a545-4d37decfba1a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:19.432310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e66265-2fd1-4690-a545-4d37decfba1a None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:19.433738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 496/1003] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:19.438777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9685b2-5814-4aef-a5dc-cfbfe4959e8f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:19.439173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9685b2-5814-4aef-a5dc-cfbfe4959e8f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:19.439362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9685b2-5814-4aef-a5dc-cfbfe4959e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:19.457213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8d9685b2-5814-4aef-a5dc-cfbfe4959e8f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:19.458428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8d9685b2-5814-4aef-a5dc-cfbfe4959e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:19.461915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:19.528054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 508/1004] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:19.551839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e66265-2fd1-4690-a545-4d37decfba1a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:19.638191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:19.663906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e66265-2fd1-4690-a545-4d37decfba1a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:19.699218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14efd0d-c2fc-490a-9cae-8d325312f3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:19.700011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14efd0d-c2fc-490a-9cae-8d325312f3ec None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:19.700297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14efd0d-c2fc-490a-9cae-8d325312f3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:19.700573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14efd0d-c2fc-490a-9cae-8d325312f3ec None admin] PATH_INFO: `/v3/services/image` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:19.701222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14efd0d-c2fc-490a-9cae-8d325312f3ec None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:19.702626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14efd0d-c2fc-490a-9cae-8d325312f3ec None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-f14efd0d-c2fc-490a-9cae-8d325312f3ec None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:19.714985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:19.716073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:19.716981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:19.725099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 497/1005] 10.222.0.35 () {56 vars in 1105 bytes} [Tue May 24 09:54:19 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:19.725306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9685b2-5814-4aef-a5dc-cfbfe4959e8f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:19.782321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e795d-2d33-4905-ac87-90dbf37a8a13 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:19.782686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e795d-2d33-4905-ac87-90dbf37a8a13 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:19.782885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e795d-2d33-4905-ac87-90dbf37a8a13 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:19.785064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 498/1006] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:19.790055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f242-41a9-42dc-ac9a-e4d0c2c16df3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:19.790348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f242-41a9-42dc-ac9a-e4d0c2c16df3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:19.790679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183f242-41a9-42dc-ac9a-e4d0c2c16df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:19.807564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5183f242-41a9-42dc-ac9a-e4d0c2c16df3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:19.808750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5183f242-41a9-42dc-ac9a-e4d0c2c16df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:19.829606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9685b2-5814-4aef-a5dc-cfbfe4959e8f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:19.868243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:19.868916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:19.869203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:19.869517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:19.870249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] RBAC: Adding query filter params (name=image) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:19.870544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:19.871935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:19.877227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 509/1007] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:19.881337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 499/1008] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:19.882866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:19.888835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:19.903480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183f242-41a9-42dc-ac9a-e4d0c2c16df3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:19.978730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:19.990685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7bed5-3717-4618-a012-5cefa2f70ffd None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:20.012167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183f242-41a9-42dc-ac9a-e4d0c2c16df3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:20.030636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:20.031304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.031585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.031863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.033087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] RBAC: Adding query filter params (type=image) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:20.033424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:20.035323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:20.039921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 510/1009] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:19 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:20.054755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d79df-3183-40bf-acb8-6c170059a913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:20.055584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d79df-3183-40bf-acb8-6c170059a913 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.055869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d79df-3183-40bf-acb8-6c170059a913 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.056144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d79df-3183-40bf-acb8-6c170059a913 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.057461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d79df-3183-40bf-acb8-6c170059a913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:20.058979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d79df-3183-40bf-acb8-6c170059a913 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:20.060930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 500/1010] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:20.067438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:20.145990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:20.183471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c2f82-d661-46c9-a7af-310483b9526e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:20.221318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:20.222576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.222861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.223152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.223958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:20.225456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:20.297047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594e8f7-6876-4820-bcd1-13092919530b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.297261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594e8f7-6876-4820-bcd1-13092919530b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.297550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594e8f7-6876-4820-bcd1-13092919530b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.298905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 501/1011] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:20.304629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3a71c-a4a2-4be4-8348-08786fe7e8c0 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.304987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3a71c-a4a2-4be4-8348-08786fe7e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.305360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3a71c-a4a2-4be4-8348-08786fe7e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.315671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 511/1012] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:54:20 2022] POST /identity/v3/projects => generated 311 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:20.323932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-32f3a71c-a4a2-4be4-8348-08786fe7e8c0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:20.325392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-32f3a71c-a4a2-4be4-8348-08786fe7e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:20.405613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 502/1013] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:20.429439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:20.539189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086422d-f13d-4119-9984-47e9f13b1c79 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:20.562786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:20.574609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:20.576366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.576366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.576366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.576708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:20.578119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:20.591250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:20.592278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:20.593451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 512/1014] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:20.601918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3a71c-a4a2-4be4-8348-08786fe7e8c0 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:20.672924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:20.710765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3a71c-a4a2-4be4-8348-08786fe7e8c0 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:20.745070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:20.745765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.746047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.746320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.747024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:20.747326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:20.748688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:20.758789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 503/1015] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:54:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:20.760057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:20.862243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe29f-18c7-421c-bbd8-13b733fc1731 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:20.898450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da58cd-3434-4ddd-a9b8-0d2c8fafdeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:20.899117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da58cd-3434-4ddd-a9b8-0d2c8fafdeaa None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:20.899396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da58cd-3434-4ddd-a9b8-0d2c8fafdeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:20.899667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da58cd-3434-4ddd-a9b8-0d2c8fafdeaa None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:20.900377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da58cd-3434-4ddd-a9b8-0d2c8fafdeaa None admin] RBAC: Adding query filter params (type=compute) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:20.900678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da58cd-3434-4ddd-a9b8-0d2c8fafdeaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:20.902079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da58cd-3434-4ddd-a9b8-0d2c8fafdeaa None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:20.913152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 513/1016] 10.222.0.35 () {56 vars in 1112 bytes} [Tue May 24 09:54:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:20.914155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:21.014192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025d491-5701-4c8e-baf8-d3e15ae444a6 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:21.043697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:21.049053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73544c68-84bf-4c2a-9565-51ab14dbfcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:21.049737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73544c68-84bf-4c2a-9565-51ab14dbfcf9 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:21.050175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73544c68-84bf-4c2a-9565-51ab14dbfcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:21.050443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73544c68-84bf-4c2a-9565-51ab14dbfcf9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:21.051373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73544c68-84bf-4c2a-9565-51ab14dbfcf9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:21.052726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73544c68-84bf-4c2a-9565-51ab14dbfcf9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:21.161764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 504/1017] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:54:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:22.310670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52254d9f-fb4d-4f75-bcc8-3bf07cc92248 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:22.311218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52254d9f-fb4d-4f75-bcc8-3bf07cc92248 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:22.311218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52254d9f-fb4d-4f75-bcc8-3bf07cc92248 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:22.312624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 514/1018] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:22.318546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b6cf7-2985-4392-97e8-b2d8a7bab42b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:22.318992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b6cf7-2985-4392-97e8-b2d8a7bab42b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:22.319091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b6cf7-2985-4392-97e8-b2d8a7bab42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:22.336786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-bb1b6cf7-2985-4392-97e8-b2d8a7bab42b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:22.337987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bb1b6cf7-2985-4392-97e8-b2d8a7bab42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:22.419405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 505/1019] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:22 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:22.442698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52254d9f-fb4d-4f75-bcc8-3bf07cc92248 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:22.496658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:22.550021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52254d9f-fb4d-4f75-bcc8-3bf07cc92248 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:22.584663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:22.585363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:22.585655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:22.585937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:22.586663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:22.588046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:22.588910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4cd5f-894f-424f-87ec-db886984934c None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:22.589198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4cd5f-894f-424f-87ec-db886984934c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:22.589524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4cd5f-894f-424f-87ec-db886984934c None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:22.591010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 506/1020] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:22.595863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834bf2-6e4a-4c11-8373-76a441733179 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:22.596150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834bf2-6e4a-4c11-8373-76a441733179 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:22.596428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17834bf2-6e4a-4c11-8373-76a441733179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:22.613838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-17834bf2-6e4a-4c11-8373-76a441733179 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:22.615001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-17834bf2-6e4a-4c11-8373-76a441733179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:22.646115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 515/1021] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:54:22 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:22.709819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 507/1022] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:22.734917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:22.848401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6635b-2c8c-40fe-829f-173b9fe32842 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:22.857165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:22.883523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a8c635-f31e-4a03-b202-dde8868bdd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:22.884327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c635-f31e-4a03-b202-dde8868bdd53 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:22.884608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c635-f31e-4a03-b202-dde8868bdd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:22.884880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a8c635-f31e-4a03-b202-dde8868bdd53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:22.886087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8c635-f31e-4a03-b202-dde8868bdd53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:22.887603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a8c635-f31e-4a03-b202-dde8868bdd53 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:22.889463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 516/1023] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:22.895294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17834bf2-6e4a-4c11-8373-76a441733179 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:22.939073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:23.010816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17834bf2-6e4a-4c11-8373-76a441733179 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:23.046778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4330596-99eb-40b4-ae78-4afe56827a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:23.047453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330596-99eb-40b4-ae78-4afe56827a96 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:23.047742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330596-99eb-40b4-ae78-4afe56827a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:23.048022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4330596-99eb-40b4-ae78-4afe56827a96 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:23.048751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4330596-99eb-40b4-ae78-4afe56827a96 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:23.050907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4330596-99eb-40b4-ae78-4afe56827a96 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:23.164957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 508/1024] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:54:22 2022] POST /identity/v3/users => generated 285 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:54:23.485496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b242-112c-46a6-a478-60431b35a05b None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:23.486022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b242-112c-46a6-a478-60431b35a05b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:23.486103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9b242-112c-46a6-a478-60431b35a05b None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:23.487517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 517/1025] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:23.493196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd9ad-b073-4ee7-8ccd-d732e275b2fa None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:23.493637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd9ad-b073-4ee7-8ccd-d732e275b2fa None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:23.493820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd9ad-b073-4ee7-8ccd-d732e275b2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:23.510820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-df8fd9ad-b073-4ee7-8ccd-d732e275b2fa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:23.511988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-df8fd9ad-b073-4ee7-8ccd-d732e275b2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:23.580701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 509/1026] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:23.605438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b242-112c-46a6-a478-60431b35a05b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:23.713388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9b242-112c-46a6-a478-60431b35a05b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:23.747772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:23.748528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:23.748806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:23.749075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:23.759556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:23.767599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:23.768528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:23.769705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:23.774269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 518/1027] 10.222.0.35 () {56 vars in 1120 bytes} [Tue May 24 09:54:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:23.776825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fd9ad-b073-4ee7-8ccd-d732e275b2fa None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:23.806391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:23.883194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fd9ad-b073-4ee7-8ccd-d732e275b2fa None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:23.919371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:23.920046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:23.920325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:23.920596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:23.921425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:23.921738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:23.923203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:23.937126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 510/1028] 10.222.0.35 () {56 vars in 1115 bytes} [Tue May 24 09:54:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:23.938964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:23.978003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:24.044774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721e344-fd50-4359-84bd-2a3e2a527729 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:24.079613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:24.080414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:24.080699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:24.080975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:24.092185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:24.093673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:24.111606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:24.112566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:24.113912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:24.113912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:24.113912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:24.113912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:24.117586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 519/1029] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:24.118873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:24.216223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:24.229935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8d58d-3d8a-4c9b-b723-8578a10053be None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:24.264677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:24.265374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:24.265662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:24.265940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:24.266665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:24.266977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:24.268342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:24.281169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 511/1030] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:24.288184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:24.382655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:24.396730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7755fd-398b-4090-a899-195d47b83b2a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:24.432986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:24.433825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:24.434110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:24.434389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:24.442206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:24.443669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:24.459640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:24.460667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:54:24.462080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:24.465337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 520/1031] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:54:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:24.466916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:24.570053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42f3cf-711e-45e0-8543-9059b285013a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:24.604175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36818a27-b574-4fa7-9d99-56aa10e9f205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:24.604857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818a27-b574-4fa7-9d99-56aa10e9f205 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:24.605137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818a27-b574-4fa7-9d99-56aa10e9f205 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:24.605466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36818a27-b574-4fa7-9d99-56aa10e9f205 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:24.606308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36818a27-b574-4fa7-9d99-56aa10e9f205 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:24.606620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36818a27-b574-4fa7-9d99-56aa10e9f205 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:24.608051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36818a27-b574-4fa7-9d99-56aa10e9f205 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:24.631402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 512/1032] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:54:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:24.632781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:24.739659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0d6f0-5ed5-4af7-ac6d-9f29514d160c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:24.776096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:24.776740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:24.777018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:24.777323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:24.795386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:24.846312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:24.847990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:24.898950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d4409-a1a4-4f53-afcd-af0d43b72812 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:24.899172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d4409-a1a4-4f53-afcd-af0d43b72812 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:24.899460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0d4409-a1a4-4f53-afcd-af0d43b72812 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:24.900959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 513/1033] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:24.905963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9af7f8-d3c4-46ad-97bd-9d4c2fbc6b9f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:24.906782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9af7f8-d3c4-46ad-97bd-9d4c2fbc6b9f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:24.907074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9af7f8-d3c4-46ad-97bd-9d4c2fbc6b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:24.919060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:24.924792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4a9af7f8-d3c4-46ad-97bd-9d4c2fbc6b9f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:24.926322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4a9af7f8-d3c4-46ad-97bd-9d4c2fbc6b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:24.951830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 521/1034] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:24.958183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:24.998979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 514/1035] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:25.006960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.022490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9af7f8-d3c4-46ad-97bd-9d4c2fbc6b9f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.061564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a536f59-4a5e-45ab-9a9a-621226353ec7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.097060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.097838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.098127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.098405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.109232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.110722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:25.129110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:25.130380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:25.131289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:25.131289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:25.131289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:25.131289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:25.131289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 522/1036] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:25.135902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.145464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9af7f8-d3c4-46ad-97bd-9d4c2fbc6b9f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.186724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.187520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.187801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.188074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] PATH_INFO: `/v3/services/image` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.188725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.190123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.193707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:25.203334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:25.208661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:25.209655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 515/1037] 10.222.0.35 () {56 vars in 1105 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/services/image => generated 86 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:25.211216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.248542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.252103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3afcc-5a69-4c0f-9b3b-01a00dd6e14d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.290570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.291254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.291535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.291808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.292548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:25.292853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.294302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.307154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 523/1038] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:25.312668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.328717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc661e7e-c4cf-449f-8ead-2178aaa33d6e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.364153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.364976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.365255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.365558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.366269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] RBAC: Adding query filter params (name=image) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:25.366575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.367936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.373296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 516/1039] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:25.378365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.424397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e8236b-d1a3-47d8-a349-fba40b12f5d5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.442288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.465357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256055a-a811-4156-b751-85ad6ed78f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.466151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256055a-a811-4156-b751-85ad6ed78f01 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.466434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256055a-a811-4156-b751-85ad6ed78f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.466709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256055a-a811-4156-b751-85ad6ed78f01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.474451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256055a-a811-4156-b751-85ad6ed78f01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.475893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256055a-a811-4156-b751-85ad6ed78f01 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.481427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dc5901-545c-4367-a0ea-f858ee56a937 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9256055a-a811-4156-b751-85ad6ed78f01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:25.491431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:25.492515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:25.493657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 524/1040] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:25.494167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934638-8284-4796-bfbc-afb3c0e1c961 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.494444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934638-8284-4796-bfbc-afb3c0e1c961 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.494719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934638-8284-4796-bfbc-afb3c0e1c961 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.496105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 525/1041] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:25.499658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d934638-8284-4796-bfbc-afb3c0e1c961 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.517097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b701ec1b-b100-4d99-829e-44fd007f2cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.517783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701ec1b-b100-4d99-829e-44fd007f2cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.518064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701ec1b-b100-4d99-829e-44fd007f2cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.518336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b701ec1b-b100-4d99-829e-44fd007f2cc0 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.519617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701ec1b-b100-4d99-829e-44fd007f2cc0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:25.519923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701ec1b-b100-4d99-829e-44fd007f2cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.521124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.521323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b701ec1b-b100-4d99-829e-44fd007f2cc0 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.526296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 517/1042] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:25.527452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be74b2-19fc-4041-9673-969bebb8a555 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.527734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be74b2-19fc-4041-9673-969bebb8a555 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.528008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be74b2-19fc-4041-9673-969bebb8a555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.544602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-03be74b2-19fc-4041-9673-969bebb8a555 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:25.545797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-03be74b2-19fc-4041-9673-969bebb8a555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:25.602012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d934638-8284-4796-bfbc-afb3c0e1c961 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.610441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 518/1043] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:25.611506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be74b2-19fc-4041-9673-969bebb8a555 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.637935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.638598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.638880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.639163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.639960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:25.640270 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.641737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.653934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 526/1044] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:25.654797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.674769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.716941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be74b2-19fc-4041-9673-969bebb8a555 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.736093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.750160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.750822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.751105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.751380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.752172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] RBAC: Adding query filter params (interface=public, service_id=5bf7cb2f3cb646ab9a58170d513d2e9d, region_id=RegionOne) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:25.752483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.754090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.754583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd863f-421d-4d65-92d6-732e506c59bf None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.759302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 519/1045] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/endpoints?service_id=5bf7cb2f3cb646ab9a58170d513d2e9d&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:25.760079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.790044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.790850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.791133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.791407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.801431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:25.808974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:25.810042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:25.811396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 527/1046] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:25.813223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.859904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a9197-cc8b-4eb5-8f10-a06d12680243 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.882104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:25.893098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.894852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.895140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.895417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.896422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] RBAC: Adding query filter params (role.id=e46f6b11f37648bba7c2f9d6e5bf7091, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=8de75a1df6784111aabec60da40ab756) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:25.896730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.898183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.903581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 520/1047] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/role_assignments?user.id=8de75a1df6784111aabec60da40ab756&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e46f6b11f37648bba7c2f9d6e5bf7091 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:25.904351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:25.924106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbca6d5-bfb7-4e70-a2a1-341236b6cd8d None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:25.968357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:25.969056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:25.969398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:25.969857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:25.971348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:25.974442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:25.983227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 528/1048] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/services => generated 1957 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 09:54:26.040817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f4a36-aad6-498f-93d3-20dab1dc87ba None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:26.082229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:26.082925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:26.083208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:26.083485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:26.084266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:26.084583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:26.086091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:26.097885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 521/1049] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:54:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:26.103934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:26.108237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:26.227662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0d6fb-2d71-45fb-9a53-132cb99abd82 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:26.263271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:26.264439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:26.264874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:26.265333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] PATH_INFO: `/v3/users/a3a57f0c9bf74f79b25ab3efde4236fa` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:26.280642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] RBAC: Authorizing `identity:get_user(user_id=a3a57f0c9bf74f79b25ab3efde4236fa)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:26.282391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:26.285245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 529/1050] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:26 2022] GET /identity/v3/users/a3a57f0c9bf74f79b25ab3efde4236fa => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:26.290713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:26.342611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:26.399976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5634f4c-dc4a-4fe2-acf1-311563f53eb2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:26.436867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:26.437832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:26.438269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:26.438701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:26.509348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:26.511017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:26.559623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:26.618762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 522/1051] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:26 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 09:54:26.620981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:26.687252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:26.727573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0d363-eb7d-41d6-aa88-170e22cbb16e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:26.762606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a3cc3-e3f6-434e-936c-652b2a1332ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:26.763388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3cc3-e3f6-434e-936c-652b2a1332ce None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:26.763667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3cc3-e3f6-434e-936c-652b2a1332ce None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:26.763975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a3cc3-e3f6-434e-936c-652b2a1332ce None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:26.765095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3cc3-e3f6-434e-936c-652b2a1332ce None admin] RBAC: Authorizing `identity:get_project(project_id=d5a5f88a877b446cbf020d56f692fe54)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:26.766630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883a3cc3-e3f6-434e-936c-652b2a1332ce None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:26.774337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 530/1052] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:26 2022] GET /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:26.775688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:26.876911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaed1d-6dc5-439f-a979-5da478c86889 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:26.912027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be98ec-5a6e-4694-a6d9-71775bad46cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:26.912687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be98ec-5a6e-4694-a6d9-71775bad46cd None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:26.912969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be98ec-5a6e-4694-a6d9-71775bad46cd None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:26.913246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be98ec-5a6e-4694-a6d9-71775bad46cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:26.914109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be98ec-5a6e-4694-a6d9-71775bad46cd None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=d5a5f88a877b446cbf020d56f692fe54, user.id=a3a57f0c9bf74f79b25ab3efde4236fa) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:26.914423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be98ec-5a6e-4694-a6d9-71775bad46cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:26.915851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be98ec-5a6e-4694-a6d9-71775bad46cd None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:26.922420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 523/1053] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:26 2022] GET /identity/v3/role_assignments?user.id=a3a57f0c9bf74f79b25ab3efde4236fa&scope.project.id=d5a5f88a877b446cbf020d56f692fe54&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:28.130125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0feea-b401-4a4e-8f98-ca18c682ead8 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.130768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0feea-b401-4a4e-8f98-ca18c682ead8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.130768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0feea-b401-4a4e-8f98-ca18c682ead8 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.132191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 531/1054] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:28.137213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687208d-34ed-4667-8232-ccdcbc77c39a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.137656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687208d-34ed-4667-8232-ccdcbc77c39a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.137805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687208d-34ed-4667-8232-ccdcbc77c39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.157745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b687208d-34ed-4667-8232-ccdcbc77c39a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:28.159184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b687208d-34ed-4667-8232-ccdcbc77c39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:28.169666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:28.233306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 524/1055] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:28.234114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225584be-a194-49ac-85d5-59d9dbfe05ee None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.234403 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225584be-a194-49ac-85d5-59d9dbfe05ee None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.234869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225584be-a194-49ac-85d5-59d9dbfe05ee None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.236241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 532/1056] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:28.241804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206303d-0254-4f6e-b5b0-f61d9bb1f3f3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.242206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206303d-0254-4f6e-b5b0-f61d9bb1f3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.242494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206303d-0254-4f6e-b5b0-f61d9bb1f3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.254290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225584be-a194-49ac-85d5-59d9dbfe05ee None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.260653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4206303d-0254-4f6e-b5b0-f61d9bb1f3f3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:28.261870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4206303d-0254-4f6e-b5b0-f61d9bb1f3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:28.336306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 525/1057] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:28 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:28.359202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206303d-0254-4f6e-b5b0-f61d9bb1f3f3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.360128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225584be-a194-49ac-85d5-59d9dbfe05ee None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:28.391038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:28.397635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:28.398452 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.398736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.399011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.410190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:28.418504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:28.419598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:28.420532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:28.424738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 533/1058] 10.222.0.35 () {56 vars in 1120 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:28.425628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.437539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:28.468677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206303d-0254-4f6e-b5b0-f61d9bb1f3f3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:28.505936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:28.506730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.507012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.507285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.507938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:28.509322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:28.522162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:28.523264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:28.524148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 526/1059] 10.222.0.35 () {56 vars in 1105 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:28.529209 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.537736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9bd99-c126-4560-893b-021483cd34b0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:28.575465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:28.576135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.576614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.576893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.577676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:28.578166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:28.579544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:28.588820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 534/1060] 10.222.0.35 () {56 vars in 1115 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:28.594406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.639750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48807e1b-edea-4ec8-a841-2bd3b39810a1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:28.652269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:28.676651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:28.677348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.677640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.677912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.678805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] RBAC: Adding query filter params (name=image) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:28.679119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:28.680491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:28.682636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:28.686729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 527/1061] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:28.692706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.708504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0eddd-701c-466a-8294-deb0dfaa3ab5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:28.745895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:28.746806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.747088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.747364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.759624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:28.761081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:28.778241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:28.779288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:28.780195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:28.780195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:28.780195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:28.780195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:28.780195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 535/1062] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:28.785095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.809509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cb79c-38df-4fbb-b678-05ec7b615f9f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:28.846556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:28.847479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.847824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.848106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.848820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] RBAC: Adding query filter params (type=image) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:28.849134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:28.850538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:28.855715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 528/1063] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:28.861032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.902927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d12475-3fb0-4fe0-8892-3dbcfe5eef74 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:28.905173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:28.930156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:28.938226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:28.938916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:28.939201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:28.939479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:28.940251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:28.940540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:28.942596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:28.961089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 536/1064] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:28.962520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:28.976171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8915e51e-041a-4e19-923d-dcf67b76fc81 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.014711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266b9cc9-88bf-4c86-b2fb-c63b20d12e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.015366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9cc9-88bf-4c86-b2fb-c63b20d12e34 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.015692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9cc9-88bf-4c86-b2fb-c63b20d12e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.015972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266b9cc9-88bf-4c86-b2fb-c63b20d12e34 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.016831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b9cc9-88bf-4c86-b2fb-c63b20d12e34 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.018352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266b9cc9-88bf-4c86-b2fb-c63b20d12e34 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.067308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9f3d4-6674-4ffe-8c40-495ddbfe8cb7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.098143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 529/1065] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:54:28 2022] POST /identity/v3/endpoints => generated 327 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:54:29.109350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.110179 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.110462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.110736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] PATH_INFO: `/v3/users/nova` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.120615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.122146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:29.137884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:29.139399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:29.140407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 537/1066] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:54:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:29.153368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bda1d-c6f4-44aa-91dd-9b97c6207f3e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.153368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bda1d-c6f4-44aa-91dd-9b97c6207f3e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.153368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30bda1d-c6f4-44aa-91dd-9b97c6207f3e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.153368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 530/1067] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:29 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:29.154418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.156064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03d0fc-e4d5-487b-ad01-eb486356a0c1 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.156354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03d0fc-e4d5-487b-ad01-eb486356a0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.156631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03d0fc-e4d5-487b-ad01-eb486356a0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.176953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:29.182875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8d03d0fc-e4d5-487b-ad01-eb486356a0c1 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:29.184121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8d03d0fc-e4d5-487b-ad01-eb486356a0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:29.258678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 531/1068] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:29.270518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bea75d8-2ecc-4e60-9ddd-33391880aaca None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.281112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03d0fc-e4d5-487b-ad01-eb486356a0c1 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.311297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.312661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.312661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.312661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.313385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:29.314491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.315123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.337991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 538/1069] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:29.342783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:29.343985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.393067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03d0fc-e4d5-487b-ad01-eb486356a0c1 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.429242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.429471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:29.430473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.430798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.431025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.442165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:29.449865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:29.450833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:29.451686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 532/1070] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:29.455466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f7070-d193-42d4-9a9e-d6c94f8ee6fc None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.457807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.493407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.494316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.494725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.495006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.506116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.507580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:29.527358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:29.528375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:29.529348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:29.529348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:29.529348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:29.529348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:29.529348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 539/1071] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:29.535038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.565073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31db82-efe3-4aa4-a5e0-ba1bf46c677e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.576326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:29.601568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.602209 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.602493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.602770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.603522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:29.603910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.605298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.615079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 533/1072] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:29.620799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.649216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7859dd5-4568-4697-b03a-5c9061fef152 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.678727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:29.685256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.685893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.686180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.686614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.687252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:29.687488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.688862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.701723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 540/1073] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:29.707754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.732186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99163dd-b02c-452a-87ad-f3b55898c311 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.766290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.767146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.767420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.767689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] PATH_INFO: `/v3/users/a3a57f0c9bf74f79b25ab3efde4236fa` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.769860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a57f0c9bf74f79b25ab3efde4236fa)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.771447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.773063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 534/1074] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/users/a3a57f0c9bf74f79b25ab3efde4236fa => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:29.778620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.820620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ef25c-64c5-43ef-8be0-2939f4bb30c1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.823886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:29.857351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.858773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.859247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.859709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.867863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.869602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:29.883949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:29.884898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:29.886192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:29.886656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a041d1d-5be8-4096-a8b1-b075aed0cde5 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:29.890093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 541/1075] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:29.890835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.922103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:29.922968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:29.923250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:29.923530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:29.924701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] RBAC: Authorizing `identity:get_project(project_id=d5a5f88a877b446cbf020d56f692fe54)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:29.926420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:29.928422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 535/1076] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:29.933736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:29.950844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:30.002894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb112701-338d-4b4d-b0c4-82aab99bcb5e None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:30.043831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:30.045298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:30.045298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:30.045298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:30.045861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:30.046167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:30.047601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:30.055395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391922e-d466-42d5-9e1c-826db39ecc5e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:30.067729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 542/1077] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:54:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:30.070843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:30.072311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:30.102843 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cd8af7-6531-4b94-8365-c3534e934565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:30.103725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8af7-6531-4b94-8365-c3534e934565 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:30.104006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8af7-6531-4b94-8365-c3534e934565 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:30.104281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cd8af7-6531-4b94-8365-c3534e934565 None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54/users/a3a57f0c9bf74f79b25ab3efde4236fa/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:30.106257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd8af7-6531-4b94-8365-c3534e934565 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5a5f88a877b446cbf020d56f692fe54, user_id=a3a57f0c9bf74f79b25ab3efde4236fa, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:30.107739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cd8af7-6531-4b94-8365-c3534e934565 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:30.177930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583faf40-3933-45e6-8de0-fe8d61cc8912 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:30.178868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 536/1078] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:54:29 2022] PUT /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54/users/a3a57f0c9bf74f79b25ab3efde4236fa/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:54:30.190593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:30.218627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b7512d-51cc-4ee4-91b9-1b71b0d444e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:30.219458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7512d-51cc-4ee4-91b9-1b71b0d444e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:30.219742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7512d-51cc-4ee4-91b9-1b71b0d444e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:30.220037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b7512d-51cc-4ee4-91b9-1b71b0d444e1 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/8de75a1df6784111aabec60da40ab756/roles/e46f6b11f37648bba7c2f9d6e5bf7091` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:30.222945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7512d-51cc-4ee4-91b9-1b71b0d444e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=8de75a1df6784111aabec60da40ab756, role_id=e46f6b11f37648bba7c2f9d6e5bf7091)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:30.224440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b7512d-51cc-4ee4-91b9-1b71b0d444e1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:30.337528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 543/1079] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:54:30 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/8de75a1df6784111aabec60da40ab756/roles/e46f6b11f37648bba7c2f9d6e5bf7091 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:54:31.354129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccfd1e-acf8-4097-a790-93370e4e3c81 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:31.354655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccfd1e-acf8-4097-a790-93370e4e3c81 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:31.354655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccfd1e-acf8-4097-a790-93370e4e3c81 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:31.356035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 537/1080] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:31.361213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e24dea-dffe-4a81-9514-1602845b861a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:31.361654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e24dea-dffe-4a81-9514-1602845b861a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:31.361802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e24dea-dffe-4a81-9514-1602845b861a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:31.379302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-82e24dea-dffe-4a81-9514-1602845b861a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:31.380487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-82e24dea-dffe-4a81-9514-1602845b861a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:31.453545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 544/1081] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:31.474652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccfd1e-acf8-4097-a790-93370e4e3c81 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:31.576719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ccfd1e-acf8-4097-a790-93370e4e3c81 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:31.610449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:31.611096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:31.611377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:31.611658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:31.646716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:31.681624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:31.683070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:31.799032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 538/1082] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:31.800768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e24dea-dffe-4a81-9514-1602845b861a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:31.895902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:31.899528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e24dea-dffe-4a81-9514-1602845b861a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:31.933797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a38f2-785e-47de-a7c6-6cc023b9020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:31.934594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a38f2-785e-47de-a7c6-6cc023b9020f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:31.934964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a38f2-785e-47de-a7c6-6cc023b9020f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:31.935245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a38f2-785e-47de-a7c6-6cc023b9020f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:31.945527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a38f2-785e-47de-a7c6-6cc023b9020f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:31.947015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a38f2-785e-47de-a7c6-6cc023b9020f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c27a38f2-785e-47de-a7c6-6cc023b9020f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:31.966047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:31.967006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:31.976982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:31.976982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:31.976982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:31.976982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:31.976982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 545/1083] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:31.977376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:32.044430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:32.079616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f5b61-c447-4478-9257-d6fa3843213c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:32.114397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642d4ef-1b70-4c5a-af11-a1c020a07dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:32.115064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d4ef-1b70-4c5a-af11-a1c020a07dec None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.115348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d4ef-1b70-4c5a-af11-a1c020a07dec None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.115626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642d4ef-1b70-4c5a-af11-a1c020a07dec None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.116354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642d4ef-1b70-4c5a-af11-a1c020a07dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:32.116665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642d4ef-1b70-4c5a-af11-a1c020a07dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:32.118054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642d4ef-1b70-4c5a-af11-a1c020a07dec None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:32.137315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 539/1084] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:32.408558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b120fcd-9a1a-47b9-94ee-1015ce65481a None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.409042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b120fcd-9a1a-47b9-94ee-1015ce65481a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.409042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b120fcd-9a1a-47b9-94ee-1015ce65481a None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.410955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 546/1085] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:32.416257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674bca-33fe-4ab2-8389-acfc6cff24fa None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.416687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674bca-33fe-4ab2-8389-acfc6cff24fa None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.416842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5674bca-33fe-4ab2-8389-acfc6cff24fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.434548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f5674bca-33fe-4ab2-8389-acfc6cff24fa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:32.435673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f5674bca-33fe-4ab2-8389-acfc6cff24fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:32.504304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 540/1086] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:32.527217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b120fcd-9a1a-47b9-94ee-1015ce65481a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:32.565255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2da4-0bbb-4392-a00a-64f77c8b742d None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.566163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2da4-0bbb-4392-a00a-64f77c8b742d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.566565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e2da4-0bbb-4392-a00a-64f77c8b742d None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.567934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 541/1087] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 09:54:32.573122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f939b-0797-44b1-bd41-26b08eba14e2 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.573450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f939b-0797-44b1-bd41-26b08eba14e2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.573734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f939b-0797-44b1-bd41-26b08eba14e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.590099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5e3f939b-0797-44b1-bd41-26b08eba14e2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:32.591278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5e3f939b-0797-44b1-bd41-26b08eba14e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:32.631212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b120fcd-9a1a-47b9-94ee-1015ce65481a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:32.643434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:32.664216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 542/1088] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:54:32.665902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:32.666737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.666987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.667260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.677701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:32.683523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f939b-0797-44b1-bd41-26b08eba14e2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:32.685171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:32.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:32.686987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 547/1089] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:54:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:32.693045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:32.707769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:32.780180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f939b-0797-44b1-bd41-26b08eba14e2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:32.791990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbeb4cc-9c9c-4391-8d40-f67ada7b1066 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:32.813991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:32.814796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.815081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.815355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.825490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:32.825658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:32.826156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.826438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.826719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.827463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:32.827770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:32.829170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:32.833781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:32.834803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:32.835647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 543/1090] 10.222.0.35 () {56 vars in 1120 bytes} [Tue May 24 09:54:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:32.837792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 548/1091] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:54:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:32.840756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:32.842911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:32.891518 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:32.918333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:32.942494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238fa9e-2f0b-4190-8020-499548abeac4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:32.945542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71ecc3-cda1-460b-aab9-0e0a982310b3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:32.976486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:32.977388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.977672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.977950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] PATH_INFO: `/v3/users/a3a57f0c9bf74f79b25ab3efde4236fa` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.978907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:32.979132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] RBAC: Authorizing `identity:get_user(user_id=a3a57f0c9bf74f79b25ab3efde4236fa)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:32.979569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:32.979852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:32.980127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:32.980596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:32.980873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:32.981182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:32.982250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 549/1092] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:32 2022] GET /identity/v3/users/a3a57f0c9bf74f79b25ab3efde4236fa => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:32.982573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:32.987887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:32.995244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 544/1093] 10.222.0.35 () {56 vars in 1115 bytes} [Tue May 24 09:54:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:32.996182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:33.087814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d9c9d-c8f2-444f-9a0c-4869396d60a6 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.094021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeddb64-67d4-4a5c-89dd-afafe0aa1d09 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.104444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:33.121517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.122171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.122453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.122732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.128981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.129983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.130207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.130481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.134997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:33.141890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.144665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:33.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:33.162400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:33.163932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:33.163932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:33.163932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:33.163932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:33.163932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 545/1094] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:33.168224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:33.193785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.195276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:33.270096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261ad83-dc56-4eff-a400-bebdc6e74224 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.292661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 550/1095] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 24 09:54:33.298390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:33.304054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.304719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.305006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.305303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.306046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:33.306349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.307717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:33.320015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 546/1096] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:33.325348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:33.340805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:33.349239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:33.408943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ea20f-83a8-4064-a87a-25f7ddbea5e4 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.437945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9804a22-e723-445a-992f-99dc1e5ae51a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.457026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.458029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.458386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.458725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.460011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] RBAC: Authorizing `identity:get_project(project_id=d5a5f88a877b446cbf020d56f692fe54)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.461807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:33.463707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 551/1097] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:33 2022] GET /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:33.469589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:33.480565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.481407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.481694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.481976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.490511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.491960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:54:33.507310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:33.508499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 24 09:54:33.509726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:33.510535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 547/1098] 10.222.0.35 () {56 vars in 1093 bytes} [Tue May 24 09:54:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:54:33.516127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:33.581095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76e0bd-0a6e-4f36-b990-6499b2ddfac6 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.593095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:33.601449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:33.621989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.622653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.622944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.623222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.624062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=d5a5f88a877b446cbf020d56f692fe54, user.id=a3a57f0c9bf74f79b25ab3efde4236fa) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:33.624375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.624575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6342561e-5bb0-4491-81c7-94b242c7af40 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.625928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:33.631982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 552/1099] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:33 2022] GET /identity/v3/role_assignments?user.id=a3a57f0c9bf74f79b25ab3efde4236fa&scope.project.id=d5a5f88a877b446cbf020d56f692fe54&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:33.669939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.670669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.670951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.671220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.672082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:33.672454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.674106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:33.696021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 548/1100] 10.222.0.35 () {56 vars in 1133 bytes} [Tue May 24 09:54:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:33.702044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:33.808195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17a6942-b9ab-46d1-8b21-3e3cbf4c445b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:33.843624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:33.844276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:33.844568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:33.844847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:33.890249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:33.915200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:33.917029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:34.013966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 553/1101] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:34.020540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:34.118211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54a126-da2b-428b-aa44-ce6049092034 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:34.152321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:34.153116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:34.153435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:34.153712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:34.163766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:34.165224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:34.166494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:54:34.184442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:34.190112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:34.191151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:54:34.191151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:54:34.191151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:54:34.191151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:34.191151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 549/1102] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:34.191550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:34.265129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:34.290325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f168a2e-804b-4567-bebf-b2709e860e6f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:34.323874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:34.324528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:34.324811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:34.325088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:34.325844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:34.326150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:34.327513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:34.339009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 554/1103] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:34.344508 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:34.442495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f2250-d7e2-442a-807e-2639c7b2ff27 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:34.475373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:34.476157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:34.476438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:34.476711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:34.483251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:34.484682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:54:34.498274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:34.499351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:54:34.500277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 550/1104] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:54:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:34.505880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:34.602945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70480c83-6be0-4e56-988b-611f00df97aa None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:34.640498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6987a-7c9f-4a8f-85c4-4b0793b3fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:34.641143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6987a-7c9f-4a8f-85c4-4b0793b3fedc None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:34.641455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6987a-7c9f-4a8f-85c4-4b0793b3fedc None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:34.641727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6987a-7c9f-4a8f-85c4-4b0793b3fedc None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:34.642505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6987a-7c9f-4a8f-85c4-4b0793b3fedc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:34.642807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6987a-7c9f-4a8f-85c4-4b0793b3fedc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:34.644277 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6987a-7c9f-4a8f-85c4-4b0793b3fedc None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:34.646125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:34.656741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 555/1105] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:54:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:34.663169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:34.714999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:34.762130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49feb7e-3fdf-46e0-bbfb-a24adbd9a23c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:34.795248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c020a-bbbf-4aca-a0df-106afac7b59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:34.795896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c020a-bbbf-4aca-a0df-106afac7b59f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:34.796178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c020a-bbbf-4aca-a0df-106afac7b59f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:34.796451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c020a-bbbf-4aca-a0df-106afac7b59f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:34.797302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c020a-bbbf-4aca-a0df-106afac7b59f None admin] RBAC: Adding query filter params (role.id=e46f6b11f37648bba7c2f9d6e5bf7091, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=8de75a1df6784111aabec60da40ab756) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:34.797677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c020a-bbbf-4aca-a0df-106afac7b59f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:34.799095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c020a-bbbf-4aca-a0df-106afac7b59f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:34.805598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 551/1106] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:34 2022] GET /identity/v3/role_assignments?user.id=8de75a1df6784111aabec60da40ab756&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e46f6b11f37648bba7c2f9d6e5bf7091 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:35.967382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e457d-5e08-4cda-82e4-cfbfdb690fd4 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:35.967942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e457d-5e08-4cda-82e4-cfbfdb690fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:35.967942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e457d-5e08-4cda-82e4-cfbfdb690fd4 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:35.969304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 556/1107] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:35.975398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4486-a7aa-4b38-9b20-0f4f7486f000 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:35.975772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4486-a7aa-4b38-9b20-0f4f7486f000 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:35.975966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b4486-a7aa-4b38-9b20-0f4f7486f000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:35.993492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-072b4486-a7aa-4b38-9b20-0f4f7486f000 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:35.994674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-072b4486-a7aa-4b38-9b20-0f4f7486f000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:36.060194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 552/1108] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:36.085738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e457d-5e08-4cda-82e4-cfbfdb690fd4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:36.187583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e457d-5e08-4cda-82e4-cfbfdb690fd4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:36.221811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae2826d-397a-464c-8d74-5e86ca5b191d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:36.223207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2826d-397a-464c-8d74-5e86ca5b191d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:36.223495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2826d-397a-464c-8d74-5e86ca5b191d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:36.223774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae2826d-397a-464c-8d74-5e86ca5b191d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:36.224960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2826d-397a-464c-8d74-5e86ca5b191d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:36.226485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae2826d-397a-464c-8d74-5e86ca5b191d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:36.228110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 557/1109] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:36.233877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4486-a7aa-4b38-9b20-0f4f7486f000 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:36.279745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:36.332101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b4486-a7aa-4b38-9b20-0f4f7486f000 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:36.365697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b98e4-65e6-42da-a678-7637627598ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:36.366357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b98e4-65e6-42da-a678-7637627598ab None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:36.366641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b98e4-65e6-42da-a678-7637627598ab None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:36.366918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b98e4-65e6-42da-a678-7637627598ab None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:36.367623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b98e4-65e6-42da-a678-7637627598ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:36.369044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b98e4-65e6-42da-a678-7637627598ab None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:36.509154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 553/1110] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:54:36 2022] POST /identity/v3/users => generated 286 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:38.732492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f801fb-a958-4507-8832-576c0d0ca371 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:38.733029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f801fb-a958-4507-8832-576c0d0ca371 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:38.733393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f801fb-a958-4507-8832-576c0d0ca371 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:38.734943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 558/1111] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:38.741426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384615c-cfba-4c4b-8677-263161cd2414 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:38.741875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384615c-cfba-4c4b-8677-263161cd2414 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:38.741967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f384615c-cfba-4c4b-8677-263161cd2414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:38.759238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f384615c-cfba-4c4b-8677-263161cd2414 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:38.760415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f384615c-cfba-4c4b-8677-263161cd2414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:38.824705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 554/1112] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:38.848729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f801fb-a958-4507-8832-576c0d0ca371 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:38.908871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:38.948069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f801fb-a958-4507-8832-576c0d0ca371 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:38.981518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:38.982366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:38.982696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:38.982922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] PATH_INFO: `/v3/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:38.984376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] RBAC: Authorizing `identity:get_role(role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:38.985788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:38.987137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 559/1113] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:38 2022] GET /identity/v3/roles/60347b7e855240eb9c04a1cb202a3500 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:38.993197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384615c-cfba-4c4b-8677-263161cd2414 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:39.023733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:39.094723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f384615c-cfba-4c4b-8677-263161cd2414 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:39.129112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:39.129997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:39.130278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:39.130550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] PATH_INFO: `/v3/users/6dcbbe6c8cfa43bf9ca59029927ecc25` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:39.143371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] RBAC: Authorizing `identity:get_user(user_id=6dcbbe6c8cfa43bf9ca59029927ecc25)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:39.145088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:39.146807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 555/1114] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:38 2022] GET /identity/v3/users/6dcbbe6c8cfa43bf9ca59029927ecc25 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:39.152477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:39.248199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b62e5-f333-4b83-a260-ab3917591f74 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:39.277849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:39.283445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:39.284086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:39.284505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:39.284861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:39.367667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:39.369101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:39.491323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 560/1115] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:39.497168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:39.583975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:39.592792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86256f4e-1507-49c2-86c9-0ec03461eeef None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:39.626173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93617917-9e50-436d-88f3-20e17a84a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:39.627018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617917-9e50-436d-88f3-20e17a84a8ed None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:39.627296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617917-9e50-436d-88f3-20e17a84a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:39.627566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93617917-9e50-436d-88f3-20e17a84a8ed None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:39.628888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93617917-9e50-436d-88f3-20e17a84a8ed None admin] RBAC: Authorizing `identity:get_project(project_id=d5a5f88a877b446cbf020d56f692fe54)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:39.630327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93617917-9e50-436d-88f3-20e17a84a8ed None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:39.631976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 556/1116] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:39 2022] GET /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:39.637970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:39.667265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:39.735830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd0cb1-d1f7-4d71-974a-4dfb73b756f0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:39.769490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c530f4a-b00e-4e64-9140-71a7711c9be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:39.770399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530f4a-b00e-4e64-9140-71a7711c9be5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:39.770689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530f4a-b00e-4e64-9140-71a7711c9be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:39.770969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c530f4a-b00e-4e64-9140-71a7711c9be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:39.771808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530f4a-b00e-4e64-9140-71a7711c9be5 None admin] RBAC: Adding query filter params (role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=d5a5f88a877b446cbf020d56f692fe54, user.id=6dcbbe6c8cfa43bf9ca59029927ecc25) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:39.772153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530f4a-b00e-4e64-9140-71a7711c9be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:39.773612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c530f4a-b00e-4e64-9140-71a7711c9be5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:39.778912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 561/1117] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:39 2022] GET /identity/v3/role_assignments?user.id=6dcbbe6c8cfa43bf9ca59029927ecc25&scope.project.id=d5a5f88a877b446cbf020d56f692fe54&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:41.943800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161ced-a050-4fb9-9291-00e56f1fe430 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:41.944337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161ced-a050-4fb9-9291-00e56f1fe430 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:41.944337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17161ced-a050-4fb9-9291-00e56f1fe430 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:41.945761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 557/1118] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:41.950915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb4ec1-6a6c-45d0-b732-ddbeede99b20 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:41.951293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb4ec1-6a6c-45d0-b732-ddbeede99b20 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:41.951468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb4ec1-6a6c-45d0-b732-ddbeede99b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:41.968488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4ccb4ec1-6a6c-45d0-b732-ddbeede99b20 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:41.969688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4ccb4ec1-6a6c-45d0-b732-ddbeede99b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:42.035494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 562/1119] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:42.057286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161ced-a050-4fb9-9291-00e56f1fe430 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:42.158200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17161ced-a050-4fb9-9291-00e56f1fe430 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:42.192915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:42.193792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:42.194132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:42.194361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] PATH_INFO: `/v3/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:42.195833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] RBAC: Authorizing `identity:get_role(role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:42.197200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:42.198613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 558/1120] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:42 2022] GET /identity/v3/roles/60347b7e855240eb9c04a1cb202a3500 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:54:42.204344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb4ec1-6a6c-45d0-b732-ddbeede99b20 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:42.227657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:42.299870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb4ec1-6a6c-45d0-b732-ddbeede99b20 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:42.333324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:42.334181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:42.334461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:42.334731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] PATH_INFO: `/v3/users/6dcbbe6c8cfa43bf9ca59029927ecc25` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:42.335904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=6dcbbe6c8cfa43bf9ca59029927ecc25)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:42.337359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:42.338971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 563/1121] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:42 2022] GET /identity/v3/users/6dcbbe6c8cfa43bf9ca59029927ecc25 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:42.344592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:42.371677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:42.443319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25da6e2-f9ce-4756-ba42-48e385ad8815 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:42.476308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47433025-c5ca-4e02-9559-90162f413d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:42.477161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433025-c5ca-4e02-9559-90162f413d62 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:42.477418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433025-c5ca-4e02-9559-90162f413d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:42.477697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433025-c5ca-4e02-9559-90162f413d62 None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:42.478857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47433025-c5ca-4e02-9559-90162f413d62 None admin] RBAC: Authorizing `identity:get_project(project_id=d5a5f88a877b446cbf020d56f692fe54)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:42.480277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47433025-c5ca-4e02-9559-90162f413d62 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:42.481974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 559/1122] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:42 2022] GET /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:42.488508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:42.576588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:42.584725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8adb4e-da4b-47e6-ab1a-325f01cfa9a7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:42.618270 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463efc6-d9c4-4a21-a5be-a54efa83a3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:42.619134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463efc6-d9c4-4a21-a5be-a54efa83a3f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:42.619413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463efc6-d9c4-4a21-a5be-a54efa83a3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:42.619686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463efc6-d9c4-4a21-a5be-a54efa83a3f6 None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54/users/6dcbbe6c8cfa43bf9ca59029927ecc25/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:42.621885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463efc6-d9c4-4a21-a5be-a54efa83a3f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5a5f88a877b446cbf020d56f692fe54, user_id=6dcbbe6c8cfa43bf9ca59029927ecc25, role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:42.623300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a463efc6-d9c4-4a21-a5be-a54efa83a3f6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:42.696543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 564/1123] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:54:42 2022] PUT /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54/users/6dcbbe6c8cfa43bf9ca59029927ecc25/roles/60347b7e855240eb9c04a1cb202a3500 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:54:44.863479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c736900-ba0b-47a7-99bd-e507009638d3 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:44.864052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c736900-ba0b-47a7-99bd-e507009638d3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:44.864052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c736900-ba0b-47a7-99bd-e507009638d3 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:44.865384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 560/1124] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:44.870646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e504b-652a-4aaa-9b52-914021c0f3c4 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:44.871029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e504b-652a-4aaa-9b52-914021c0f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:44.871232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e504b-652a-4aaa-9b52-914021c0f3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:44.888233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-bb1e504b-652a-4aaa-9b52-914021c0f3c4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:44.889442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bb1e504b-652a-4aaa-9b52-914021c0f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:44.953920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 565/1125] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:44.977529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c736900-ba0b-47a7-99bd-e507009638d3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:45.069792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:45.077105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c736900-ba0b-47a7-99bd-e507009638d3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:45.111126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:45.111966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:45.112256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:45.113072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] PATH_INFO: `/v3/roles/60347b7e855240eb9c04a1cb202a3500` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:45.114631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] RBAC: Authorizing `identity:get_role(role_id=60347b7e855240eb9c04a1cb202a3500)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:45.115948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:45.117313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 561/1126] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:44 2022] GET /identity/v3/roles/60347b7e855240eb9c04a1cb202a3500 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:45.123177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e504b-652a-4aaa-9b52-914021c0f3c4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:45.202160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:45.219245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e504b-652a-4aaa-9b52-914021c0f3c4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:45.252225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:45.253133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:45.254029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:45.254211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] PATH_INFO: `/v3/users/6dcbbe6c8cfa43bf9ca59029927ecc25` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:45.255467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] RBAC: Authorizing `identity:get_user(user_id=6dcbbe6c8cfa43bf9ca59029927ecc25)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:45.256951 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:45.258613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 566/1127] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:45 2022] GET /identity/v3/users/6dcbbe6c8cfa43bf9ca59029927ecc25 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:45.264278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:45.358662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beee56-2755-4eb2-9079-9cd2d6a8d29a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:45.391928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:45.392515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:45.392801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:45.393135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:45.422349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:45.460352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:45.461850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:45.556781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 562/1128] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:45.562789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:45.658079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47599c4d-4054-404a-a177-06fffca0da7f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:45.691406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e1eb7f-8291-4020-870d-03d23586411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:45.692255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1eb7f-8291-4020-870d-03d23586411b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:45.692672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1eb7f-8291-4020-870d-03d23586411b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:45.692887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e1eb7f-8291-4020-870d-03d23586411b None admin] PATH_INFO: `/v3/projects/d5a5f88a877b446cbf020d56f692fe54` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:45.694082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1eb7f-8291-4020-870d-03d23586411b None admin] RBAC: Authorizing `identity:get_project(project_id=d5a5f88a877b446cbf020d56f692fe54)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:45.695510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e1eb7f-8291-4020-870d-03d23586411b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:45.697167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 567/1129] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:45 2022] GET /identity/v3/projects/d5a5f88a877b446cbf020d56f692fe54 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:45.703194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:45.762104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:45.798847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e483d-f84b-4683-9bce-316455a19c9a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:45.831912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c38a21-f034-46fb-8789-ff2514ac6a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:45.832587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c38a21-f034-46fb-8789-ff2514ac6a1b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:45.832866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c38a21-f034-46fb-8789-ff2514ac6a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:45.833141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c38a21-f034-46fb-8789-ff2514ac6a1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:45.833996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c38a21-f034-46fb-8789-ff2514ac6a1b None admin] RBAC: Adding query filter params (role.id=60347b7e855240eb9c04a1cb202a3500, scope.project.id=d5a5f88a877b446cbf020d56f692fe54, user.id=6dcbbe6c8cfa43bf9ca59029927ecc25) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:45.834303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c38a21-f034-46fb-8789-ff2514ac6a1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:45.835719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c38a21-f034-46fb-8789-ff2514ac6a1b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:45.841383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 563/1130] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:45 2022] GET /identity/v3/role_assignments?user.id=6dcbbe6c8cfa43bf9ca59029927ecc25&scope.project.id=d5a5f88a877b446cbf020d56f692fe54&role.id=60347b7e855240eb9c04a1cb202a3500 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:48.063974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9b6db-adf3-467f-8c4d-4dd8fc701adc None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:48.064558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9b6db-adf3-467f-8c4d-4dd8fc701adc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:48.064558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9b6db-adf3-467f-8c4d-4dd8fc701adc None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:48.066447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 568/1131] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:48.071590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405fb4-fa5e-4246-81f1-67e1bed3862e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:48.071994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405fb4-fa5e-4246-81f1-67e1bed3862e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:48.072149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f405fb4-fa5e-4246-81f1-67e1bed3862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:48.089510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-3f405fb4-fa5e-4246-81f1-67e1bed3862e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:48.090714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3f405fb4-fa5e-4246-81f1-67e1bed3862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:48.155358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 564/1132] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:48.178792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9b6db-adf3-467f-8c4d-4dd8fc701adc None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:48.198268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:48.279494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9b6db-adf3-467f-8c4d-4dd8fc701adc None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:48.312492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87c8bc-4c7a-4a85-8189-80190baa93ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:48.313295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87c8bc-4c7a-4a85-8189-80190baa93ae None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:48.313578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87c8bc-4c7a-4a85-8189-80190baa93ae None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:48.313850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87c8bc-4c7a-4a85-8189-80190baa93ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:48.315053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87c8bc-4c7a-4a85-8189-80190baa93ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:48.317121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87c8bc-4c7a-4a85-8189-80190baa93ae None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:48.318736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 569/1133] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:48.324480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405fb4-fa5e-4246-81f1-67e1bed3862e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:48.377122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:48.419197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f405fb4-fa5e-4246-81f1-67e1bed3862e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:48.452562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f481b-5ec9-4c5f-b0b8-ca253f9b97e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:48.453211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f481b-5ec9-4c5f-b0b8-ca253f9b97e1 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:48.453523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f481b-5ec9-4c5f-b0b8-ca253f9b97e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:48.453798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f481b-5ec9-4c5f-b0b8-ca253f9b97e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:48.454511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f481b-5ec9-4c5f-b0b8-ca253f9b97e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:48.455927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f481b-5ec9-4c5f-b0b8-ca253f9b97e1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:48.560444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 565/1134] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:54:48 2022] POST /identity/v3/projects => generated 311 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:54:50.800429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d42b4-6442-4f38-b000-b8c2c9030df6 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:50.800963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d42b4-6442-4f38-b000-b8c2c9030df6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:50.800963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d42b4-6442-4f38-b000-b8c2c9030df6 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:50.802362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 570/1135] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:50.808465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78af1-d032-4102-98a5-84e1784934f7 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:50.808890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78af1-d032-4102-98a5-84e1784934f7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:50.809026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf78af1-d032-4102-98a5-84e1784934f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:50.826273 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-6bf78af1-d032-4102-98a5-84e1784934f7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:50.827448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6bf78af1-d032-4102-98a5-84e1784934f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:50.891239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 566/1136] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:50.916523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d42b4-6442-4f38-b000-b8c2c9030df6 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:51.001104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:51.016616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d42b4-6442-4f38-b000-b8c2c9030df6 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:51.051118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8115c2d4-fea5-458e-849c-f25c26b95ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:51.051945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115c2d4-fea5-458e-849c-f25c26b95ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:51.052235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115c2d4-fea5-458e-849c-f25c26b95ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:51.052513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8115c2d4-fea5-458e-849c-f25c26b95ff1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:51.053739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115c2d4-fea5-458e-849c-f25c26b95ff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:51.055180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8115c2d4-fea5-458e-849c-f25c26b95ff1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:51.056786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 571/1137] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:54:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:51.062441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf78af1-d032-4102-98a5-84e1784934f7 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:51.100959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:51.158154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf78af1-d032-4102-98a5-84e1784934f7 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:51.191858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e6d21-a4f1-4dd4-b6cd-c5e2601b98b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:51.193661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e6d21-a4f1-4dd4-b6cd-c5e2601b98b7 None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:51.193661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e6d21-a4f1-4dd4-b6cd-c5e2601b98b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:51.193661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e6d21-a4f1-4dd4-b6cd-c5e2601b98b7 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:51.193967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e6d21-a4f1-4dd4-b6cd-c5e2601b98b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:51.195197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e6d21-a4f1-4dd4-b6cd-c5e2601b98b7 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:51.359666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 567/1138] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:54:51 2022] POST /identity/v3/users => generated 286 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:54:53.574242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d050-c609-4013-aa3b-927e3f27b94e None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:53.574801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d050-c609-4013-aa3b-927e3f27b94e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:53.574801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6d050-c609-4013-aa3b-927e3f27b94e None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:53.576149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 572/1139] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:53.581847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe112c-c571-4d79-86cc-5774a6d67888 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:53.582277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe112c-c571-4d79-86cc-5774a6d67888 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:53.582397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe112c-c571-4d79-86cc-5774a6d67888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:53.599316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2fbe112c-c571-4d79-86cc-5774a6d67888 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:53.600478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2fbe112c-c571-4d79-86cc-5774a6d67888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:53.664193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 568/1140] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:53.686974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d050-c609-4013-aa3b-927e3f27b94e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:53.785237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6d050-c609-4013-aa3b-927e3f27b94e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:53.817815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:53.818605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:53.818885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:53.819159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:53.829128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:53.837220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:53.838379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:53.839426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 573/1141] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:53.844474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe112c-c571-4d79-86cc-5774a6d67888 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:53.869167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:53.941197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe112c-c571-4d79-86cc-5774a6d67888 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:53.974310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:53.974965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:53.975244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:53.975517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:53.976254 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:53.976557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:53.977941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:53.985631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 569/1142] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:54:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:53.991514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:54.008173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:54.089104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4faad3b-dd86-4079-b8a8-6ab8b2d19703 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:54.122563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:54.123430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:54.123712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:54.123985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] PATH_INFO: `/v3/users/10391fba921e42b2bc18e177dbcae354` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:54.137251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] RBAC: Authorizing `identity:get_user(user_id=10391fba921e42b2bc18e177dbcae354)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:54.138728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:54.140400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 574/1143] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:53 2022] GET /identity/v3/users/10391fba921e42b2bc18e177dbcae354 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:54.146718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:54.230572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:54.245284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d45d31-1529-4a68-be6d-c01ff5150d2b None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:54.279069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:54.279707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:54.279986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:54.280289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:54.344601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:54.345987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:54.426535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:54.441365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 570/1144] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:54:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:54:54.447201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:54.519633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:54.547248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec3439-45e6-4ae8-9521-cb751a7cc9c3 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:54.580171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccf90b4-46e7-4c8d-a535-d50ed388975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:54.581024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf90b4-46e7-4c8d-a535-d50ed388975a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:54.581994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf90b4-46e7-4c8d-a535-d50ed388975a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:54.582270 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf90b4-46e7-4c8d-a535-d50ed388975a None admin] PATH_INFO: `/v3/projects/eb33c073e6264cdd9613142eb5c980c6` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:54.583432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf90b4-46e7-4c8d-a535-d50ed388975a None admin] RBAC: Authorizing `identity:get_project(project_id=eb33c073e6264cdd9613142eb5c980c6)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:54.584857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccf90b4-46e7-4c8d-a535-d50ed388975a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:54.586557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 575/1145] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:54 2022] GET /identity/v3/projects/eb33c073e6264cdd9613142eb5c980c6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:54.592579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:54.687989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1d8b5-2c2f-47d4-881e-e2dd607b2787 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:54.720061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b50104-6d5c-4d40-b0ff-daa14f7907c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:54.720696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50104-6d5c-4d40-b0ff-daa14f7907c2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:54.720976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50104-6d5c-4d40-b0ff-daa14f7907c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:54.721251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b50104-6d5c-4d40-b0ff-daa14f7907c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:54.722643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b50104-6d5c-4d40-b0ff-daa14f7907c2 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=eb33c073e6264cdd9613142eb5c980c6, user.id=10391fba921e42b2bc18e177dbcae354) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:54.723114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b50104-6d5c-4d40-b0ff-daa14f7907c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:54.724550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b50104-6d5c-4d40-b0ff-daa14f7907c2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:54.729847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 571/1146] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:54:54 2022] GET /identity/v3/role_assignments?user.id=10391fba921e42b2bc18e177dbcae354&scope.project.id=eb33c073e6264cdd9613142eb5c980c6&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:56.885414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c783-2b82-4821-804b-cc8b14d343eb None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:56.885959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c783-2b82-4821-804b-cc8b14d343eb None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:56.885959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42c783-2b82-4821-804b-cc8b14d343eb None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:56.887345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 576/1147] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:54:56.892613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbacd7-3f59-428b-b3f2-a33d66fc9954 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:56.893020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbacd7-3f59-428b-b3f2-a33d66fc9954 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:56.893168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbacd7-3f59-428b-b3f2-a33d66fc9954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:56.910729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f0dbacd7-3f59-428b-b3f2-a33d66fc9954 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:54:56.911901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f0dbacd7-3f59-428b-b3f2-a33d66fc9954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:54:56.956027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:56.976713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 572/1148] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:54:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:54:57.000451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42c783-2b82-4821-804b-cc8b14d343eb None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:57.101767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42c783-2b82-4821-804b-cc8b14d343eb None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:57.136144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:57.136876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:57.137160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:57.137472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:57.145296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:57.148530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:54:57.156114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:54:57.157050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:54:57.157971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 577/1149] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:54:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:54:57.163623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbacd7-3f59-428b-b3f2-a33d66fc9954 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:57.266008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbacd7-3f59-428b-b3f2-a33d66fc9954 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:57.298244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:57.298895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:57.299173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:57.299445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:57.300739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:54:57.301089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:57.302436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:57.310181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 573/1150] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:54:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:57.316749 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:57.412555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf329-035d-4767-85b5-8ff369eb9246 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:57.445307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:57.446159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:57.446447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:57.446719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] PATH_INFO: `/v3/users/10391fba921e42b2bc18e177dbcae354` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:57.447884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] RBAC: Authorizing `identity:get_user(user_id=10391fba921e42b2bc18e177dbcae354)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:57.449362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:57.450962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 578/1151] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:54:57 2022] GET /identity/v3/users/10391fba921e42b2bc18e177dbcae354 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:57.459267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:57.505286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:57.560018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2a8e82-bb3c-4d9f-aecd-7878f81bb7d9 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:57.592795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536227a-9f9e-49cf-b8db-2810b9aa53a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:57.593674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536227a-9f9e-49cf-b8db-2810b9aa53a9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:57.593956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536227a-9f9e-49cf-b8db-2810b9aa53a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:57.594231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536227a-9f9e-49cf-b8db-2810b9aa53a9 None admin] PATH_INFO: `/v3/projects/eb33c073e6264cdd9613142eb5c980c6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:57.595392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536227a-9f9e-49cf-b8db-2810b9aa53a9 None admin] RBAC: Authorizing `identity:get_project(project_id=eb33c073e6264cdd9613142eb5c980c6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:57.596818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536227a-9f9e-49cf-b8db-2810b9aa53a9 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:57.599072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 574/1152] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:54:57 2022] GET /identity/v3/projects/eb33c073e6264cdd9613142eb5c980c6 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:54:57.604989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:54:57.669471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:54:57.701233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa84ec9-ed38-4e27-83c3-400abca90a6f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:54:57.733048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee61a53-6ce3-4e04-aecc-14dc6ae5fe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:54:57.733934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee61a53-6ce3-4e04-aecc-14dc6ae5fe9f None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:54:57.734214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee61a53-6ce3-4e04-aecc-14dc6ae5fe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:54:57.734484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee61a53-6ce3-4e04-aecc-14dc6ae5fe9f None admin] PATH_INFO: `/v3/projects/eb33c073e6264cdd9613142eb5c980c6/users/10391fba921e42b2bc18e177dbcae354/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:54:57.736376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee61a53-6ce3-4e04-aecc-14dc6ae5fe9f None admin] RBAC: Authorizing `identity:create_grant(project_id=eb33c073e6264cdd9613142eb5c980c6, user_id=10391fba921e42b2bc18e177dbcae354, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:54:57.738536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee61a53-6ce3-4e04-aecc-14dc6ae5fe9f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:54:57.851883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 579/1153] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:54:57 2022] PUT /identity/v3/projects/eb33c073e6264cdd9613142eb5c980c6/users/10391fba921e42b2bc18e177dbcae354/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:55:00.028317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4b5a-0de0-4317-ab30-7edcf2ecfba1 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:00.028917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4b5a-0de0-4317-ab30-7edcf2ecfba1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:00.028917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f4b5a-0de0-4317-ab30-7edcf2ecfba1 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:00.030257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 575/1154] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:00.035920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72584d-dbda-4f6a-b488-d2a0a3377788 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:00.036322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72584d-dbda-4f6a-b488-d2a0a3377788 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:00.036479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72584d-dbda-4f6a-b488-d2a0a3377788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:00.053517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-cb72584d-dbda-4f6a-b488-d2a0a3377788 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:00.054721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cb72584d-dbda-4f6a-b488-d2a0a3377788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:00.118324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 580/1155] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:55:00.141567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f4b5a-0de0-4317-ab30-7edcf2ecfba1 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:00.243309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f4b5a-0de0-4317-ab30-7edcf2ecfba1 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:00.254350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:00.277529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:00.278301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:00.278627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:00.278853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:00.289246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:55:00.296797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:55:00.300856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:55:00.301998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 576/1156] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:55:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:55:00.304286 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72584d-dbda-4f6a-b488-d2a0a3377788 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:00.360959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:00.401728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72584d-dbda-4f6a-b488-d2a0a3377788 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:00.434086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:00.434748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:00.435035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:00.435315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:00.436059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:00.436373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:00.437784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:00.445927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 581/1157] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:55:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:00.451407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:00.548206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b8b19-b989-4f89-995b-ff4b44a879cc None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:00.580285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:00.581122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:00.581477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:00.581750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] PATH_INFO: `/v3/users/10391fba921e42b2bc18e177dbcae354` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:00.583064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] RBAC: Authorizing `identity:get_user(user_id=10391fba921e42b2bc18e177dbcae354)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:00.584477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:00.586069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 577/1158] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:55:00 2022] GET /identity/v3/users/10391fba921e42b2bc18e177dbcae354 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:00.591765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:00.688407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b2422-3ead-4692-b267-24a8f956bfb0 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:00.707248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:00.720483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:00.726368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:00.726368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:00.726368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:00.787281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:00.788731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:00.889132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 582/1159] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:55:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:55:00.895204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:00.934233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:00.992985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba134d7-02a2-4756-adb5-784b71964b45 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:01.026819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5b20c6-4bb5-4b09-ac67-c6343bbadc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:01.027653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b20c6-4bb5-4b09-ac67-c6343bbadc21 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:01.027929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b20c6-4bb5-4b09-ac67-c6343bbadc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:01.028196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5b20c6-4bb5-4b09-ac67-c6343bbadc21 None admin] PATH_INFO: `/v3/projects/eb33c073e6264cdd9613142eb5c980c6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:01.029373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b20c6-4bb5-4b09-ac67-c6343bbadc21 None admin] RBAC: Authorizing `identity:get_project(project_id=eb33c073e6264cdd9613142eb5c980c6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:01.030793 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5b20c6-4bb5-4b09-ac67-c6343bbadc21 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:01.032439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 578/1160] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:55:00 2022] GET /identity/v3/projects/eb33c073e6264cdd9613142eb5c980c6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:01.038549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:01.059308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:01.139397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7778eaa-20d5-49f1-9311-8d2f0ada69b3 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:01.172489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aee49a-c835-4f2a-9205-bcea432f7c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:01.173138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aee49a-c835-4f2a-9205-bcea432f7c08 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:01.173451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aee49a-c835-4f2a-9205-bcea432f7c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:01.173728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aee49a-c835-4f2a-9205-bcea432f7c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:01.174551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aee49a-c835-4f2a-9205-bcea432f7c08 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=eb33c073e6264cdd9613142eb5c980c6, user.id=10391fba921e42b2bc18e177dbcae354) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:01.174859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aee49a-c835-4f2a-9205-bcea432f7c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:01.176827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aee49a-c835-4f2a-9205-bcea432f7c08 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:01.182631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 583/1161] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:55:01 2022] GET /identity/v3/role_assignments?user.id=10391fba921e42b2bc18e177dbcae354&scope.project.id=eb33c073e6264cdd9613142eb5c980c6&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:03.413915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773d58-fa42-417c-83c7-9d03abe35a5b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:03.414446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773d58-fa42-417c-83c7-9d03abe35a5b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:03.414446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35773d58-fa42-417c-83c7-9d03abe35a5b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:03.415724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 579/1162] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:03.421374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee5fc6-547f-4665-ba02-ae2f99a8e071 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:03.421837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee5fc6-547f-4665-ba02-ae2f99a8e071 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:03.421932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee5fc6-547f-4665-ba02-ae2f99a8e071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:03.438663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7cee5fc6-547f-4665-ba02-ae2f99a8e071 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:03.439927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7cee5fc6-547f-4665-ba02-ae2f99a8e071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:03.508094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 584/1163] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:55:03.538160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773d58-fa42-417c-83c7-9d03abe35a5b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:03.653227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35773d58-fa42-417c-83c7-9d03abe35a5b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:03.660860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:03.688438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:03.689093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:03.689416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:03.689695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:03.764688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:03.766236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:03.874831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 580/1164] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:55:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:55:03.882664 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee5fc6-547f-4665-ba02-ae2f99a8e071 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:03.898366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:03.990339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee5fc6-547f-4665-ba02-ae2f99a8e071 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:04.023706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6711b0b-6318-4121-87d4-bda7e8eb69b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:04.024504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6711b0b-6318-4121-87d4-bda7e8eb69b5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:04.024790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6711b0b-6318-4121-87d4-bda7e8eb69b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:04.025063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6711b0b-6318-4121-87d4-bda7e8eb69b5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:04.031632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6711b0b-6318-4121-87d4-bda7e8eb69b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:04.033074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6711b0b-6318-4121-87d4-bda7e8eb69b5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-c6711b0b-6318-4121-87d4-bda7e8eb69b5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:55:04.047209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:55:04.048458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:55:04.049537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 585/1165] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:55:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:55:04.055239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:04.063344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:04.152443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b41f7f-fda7-4dc1-b5d9-59d9480d1bc7 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:04.186789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b7c970-176b-4987-8ea6-c024b8571a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:04.187455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7c970-176b-4987-8ea6-c024b8571a0f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:04.187741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7c970-176b-4987-8ea6-c024b8571a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:04.188023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b7c970-176b-4987-8ea6-c024b8571a0f None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:04.188752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7c970-176b-4987-8ea6-c024b8571a0f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:04.189066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7c970-176b-4987-8ea6-c024b8571a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:04.190499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b7c970-176b-4987-8ea6-c024b8571a0f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:04.198253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 581/1166] 10.222.0.35 () {56 vars in 1115 bytes} [Tue May 24 09:55:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:06.367290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33185b8b-73ef-4765-8330-f5e0dc68a588 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:06.367843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33185b8b-73ef-4765-8330-f5e0dc68a588 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:06.367843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33185b8b-73ef-4765-8330-f5e0dc68a588 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:06.369213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 586/1167] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:06.374416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c004cc-22ab-4eb1-b0d0-28245c693115 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:06.374831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c004cc-22ab-4eb1-b0d0-28245c693115 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:06.374982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c004cc-22ab-4eb1-b0d0-28245c693115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:06.391521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a1c004cc-22ab-4eb1-b0d0-28245c693115 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:06.392736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a1c004cc-22ab-4eb1-b0d0-28245c693115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:06.454439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:06.459656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 582/1168] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:55:06.482633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33185b8b-73ef-4765-8330-f5e0dc68a588 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:06.556988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:06.582364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33185b8b-73ef-4765-8330-f5e0dc68a588 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:06.615175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0d437-f4ce-4745-ab68-7c2720565bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:06.615823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d437-f4ce-4745-ab68-7c2720565bc6 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:06.616102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d437-f4ce-4745-ab68-7c2720565bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:06.616376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0d437-f4ce-4745-ab68-7c2720565bc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:06.617111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0d437-f4ce-4745-ab68-7c2720565bc6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:06.618501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0d437-f4ce-4745-ab68-7c2720565bc6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:06.726886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 587/1169] 10.222.0.35 () {60 vars in 1138 bytes} [Tue May 24 09:55:06 2022] POST /identity/v3/domains => generated 267 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:55:08.907309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afe929-ed95-4070-b46f-cdc257e1be39 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:08.907857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afe929-ed95-4070-b46f-cdc257e1be39 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:08.907857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afe929-ed95-4070-b46f-cdc257e1be39 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:08.909198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 583/1170] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:08.915046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98545-32a9-4006-81ac-eeaed421620f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:08.915451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98545-32a9-4006-81ac-eeaed421620f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:08.915606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be98545-32a9-4006-81ac-eeaed421620f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:08.932806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6be98545-32a9-4006-81ac-eeaed421620f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:08.934022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6be98545-32a9-4006-81ac-eeaed421620f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:08.998847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 588/1171] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:55:09.022034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afe929-ed95-4070-b46f-cdc257e1be39 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:09.121425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afe929-ed95-4070-b46f-cdc257e1be39 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:09.154621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442e298-2699-4bc2-abdd-23e768ada71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:09.155468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442e298-2699-4bc2-abdd-23e768ada71e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:09.155748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442e298-2699-4bc2-abdd-23e768ada71e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:09.156023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442e298-2699-4bc2-abdd-23e768ada71e None admin] PATH_INFO: `/v3/domains/4f17fd20254e495e995f11b0cf85a816` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:09.166343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442e298-2699-4bc2-abdd-23e768ada71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=4f17fd20254e495e995f11b0cf85a816)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:09.167787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442e298-2699-4bc2-abdd-23e768ada71e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:09.169439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 584/1172] 10.222.0.35 () {56 vars in 1183 bytes} [Tue May 24 09:55:09 2022] GET /identity/v3/domains/4f17fd20254e495e995f11b0cf85a816 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:09.175712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be98545-32a9-4006-81ac-eeaed421620f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:09.235511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:09.276413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be98545-32a9-4006-81ac-eeaed421620f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:09.309619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341b6e3-01c7-437d-b770-07b4f00153b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:09.310279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341b6e3-01c7-437d-b770-07b4f00153b0 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:09.310587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341b6e3-01c7-437d-b770-07b4f00153b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:09.310863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341b6e3-01c7-437d-b770-07b4f00153b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:09.311581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341b6e3-01c7-437d-b770-07b4f00153b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:09.313012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341b6e3-01c7-437d-b770-07b4f00153b0 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:09.401283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 589/1173] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:55:09 2022] POST /identity/v3/projects => generated 361 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:55:11.608431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e588-990c-4ce0-b539-0956b38edf04 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:11.609006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e588-990c-4ce0-b539-0956b38edf04 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:11.609006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15e588-990c-4ce0-b539-0956b38edf04 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:11.610376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 585/1174] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:11.616122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbc109-def4-48f2-a84b-19b011759c10 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:11.616554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbc109-def4-48f2-a84b-19b011759c10 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:11.616682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbc109-def4-48f2-a84b-19b011759c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:11.633745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6bbbc109-def4-48f2-a84b-19b011759c10 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:11.634922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6bbbc109-def4-48f2-a84b-19b011759c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:11.699213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 590/1175] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:55:11.724159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15e588-990c-4ce0-b539-0956b38edf04 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:11.782021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:11.822038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15e588-990c-4ce0-b539-0956b38edf04 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:11.854105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc4929-05f0-4b3a-92be-e2dd3c4e8cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:11.854951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc4929-05f0-4b3a-92be-e2dd3c4e8cfa None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:11.855235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc4929-05f0-4b3a-92be-e2dd3c4e8cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:11.855512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc4929-05f0-4b3a-92be-e2dd3c4e8cfa None admin] PATH_INFO: `/v3/domains/4f17fd20254e495e995f11b0cf85a816` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:11.856670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc4929-05f0-4b3a-92be-e2dd3c4e8cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=4f17fd20254e495e995f11b0cf85a816)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:11.858114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc4929-05f0-4b3a-92be-e2dd3c4e8cfa None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:11.859744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 586/1176] 10.222.0.35 () {56 vars in 1183 bytes} [Tue May 24 09:55:11 2022] GET /identity/v3/domains/4f17fd20254e495e995f11b0cf85a816 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:11.866006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbc109-def4-48f2-a84b-19b011759c10 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:11.910384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:11.960598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbc109-def4-48f2-a84b-19b011759c10 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:11.993426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89c1466-ceff-4962-83c1-83bfbfeaa817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:11.994080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1466-ceff-4962-83c1-83bfbfeaa817 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:11.994359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1466-ceff-4962-83c1-83bfbfeaa817 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:11.994631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89c1466-ceff-4962-83c1-83bfbfeaa817 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:11.995341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c1466-ceff-4962-83c1-83bfbfeaa817 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:11.996773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89c1466-ceff-4962-83c1-83bfbfeaa817 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:12.095576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 591/1177] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:55:11 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:55:14.320901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84738e-9b0e-46f4-942f-71f3fb9c073e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:14.321455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84738e-9b0e-46f4-942f-71f3fb9c073e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:14.321544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c84738e-9b0e-46f4-942f-71f3fb9c073e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:14.322827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 587/1178] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:14.328177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f290f-6b75-44c5-ba33-50444170a3ab None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:14.328577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f290f-6b75-44c5-ba33-50444170a3ab None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:14.328742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f290f-6b75-44c5-ba33-50444170a3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:14.346123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-bd6f290f-6b75-44c5-ba33-50444170a3ab None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:14.348099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bd6f290f-6b75-44c5-ba33-50444170a3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:14.414418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 592/1179] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:55:14.438071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84738e-9b0e-46f4-942f-71f3fb9c073e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:14.537433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c84738e-9b0e-46f4-942f-71f3fb9c073e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:14.562458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:14.570150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:14.570925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:14.571760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:14.571760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:14.581240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:55:14.589712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:55:14.590669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:55:14.591685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 588/1180] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:55:14.597010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f290f-6b75-44c5-ba33-50444170a3ab None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:14.626884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:14.692824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f290f-6b75-44c5-ba33-50444170a3ab None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:14.726152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:14.726806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:14.727090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:14.727365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:14.728108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:14.728413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:14.729832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:14.737592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 593/1181] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:14.743353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:14.840525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a255f516-ac73-4350-a446-52fa94a0be74 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:14.872003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:14.872852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:14.873135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:14.873444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] PATH_INFO: `/v3/users/29eb89dc7369434d9b88df75a82c22b6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:14.887250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] RBAC: Authorizing `identity:get_user(user_id=29eb89dc7369434d9b88df75a82c22b6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:14.888679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:14.890308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 589/1182] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:55:14 2022] GET /identity/v3/users/29eb89dc7369434d9b88df75a82c22b6 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:14.896757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:14.987371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:14.994052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5aa54-2d1d-43ee-9835-83660396c143 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:15.030205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:15.030854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:15.031145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:15.031415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:15.095967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:15.097487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:15.187003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:15.194691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 594/1183] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:55:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:55:15.216732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:15.246914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:15.298583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94232c5c-eecf-4de3-8512-415e2a78bca6 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:15.333277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fe0a0-70a6-4be5-bc67-c558ad89a23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:15.334152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fe0a0-70a6-4be5-bc67-c558ad89a23b None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:15.334435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fe0a0-70a6-4be5-bc67-c558ad89a23b None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:15.334713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fe0a0-70a6-4be5-bc67-c558ad89a23b None admin] PATH_INFO: `/v3/projects/29143af1cad442009432ec483e4d60e0` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:15.335880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fe0a0-70a6-4be5-bc67-c558ad89a23b None admin] RBAC: Authorizing `identity:get_project(project_id=29143af1cad442009432ec483e4d60e0)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:15.337372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fe0a0-70a6-4be5-bc67-c558ad89a23b None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:15.339041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 590/1184] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:55:15 2022] GET /identity/v3/projects/29143af1cad442009432ec483e4d60e0 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:15.345151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:15.443759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc7cbee-3519-4245-be8f-71183016700b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:15.477039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e26770-13ba-47c9-a5c3-b6c7f60f1919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:15.477709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26770-13ba-47c9-a5c3-b6c7f60f1919 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:15.477990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26770-13ba-47c9-a5c3-b6c7f60f1919 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:15.478265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e26770-13ba-47c9-a5c3-b6c7f60f1919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:15.479083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26770-13ba-47c9-a5c3-b6c7f60f1919 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=29143af1cad442009432ec483e4d60e0, user.id=29eb89dc7369434d9b88df75a82c22b6) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:15.479386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26770-13ba-47c9-a5c3-b6c7f60f1919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:15.480798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e26770-13ba-47c9-a5c3-b6c7f60f1919 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:15.486075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 595/1185] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:55:15 2022] GET /identity/v3/role_assignments?user.id=29eb89dc7369434d9b88df75a82c22b6&scope.project.id=29143af1cad442009432ec483e4d60e0&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:17.664300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d21b6a-d618-4922-9fe4-813fa0436cb2 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:17.664864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d21b6a-d618-4922-9fe4-813fa0436cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:17.664864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d21b6a-d618-4922-9fe4-813fa0436cb2 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:17.666219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 591/1186] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:17.671373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1898a-7c44-4ec9-8451-4760c932747a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:17.671749 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1898a-7c44-4ec9-8451-4760c932747a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:17.671930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1898a-7c44-4ec9-8451-4760c932747a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:17.688289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-08e1898a-7c44-4ec9-8451-4760c932747a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:17.689483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-08e1898a-7c44-4ec9-8451-4760c932747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:17.740495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:17.754152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 596/1187] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:55:17.777012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d21b6a-d618-4922-9fe4-813fa0436cb2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:17.874548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d21b6a-d618-4922-9fe4-813fa0436cb2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:17.888356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:17.907958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:17.908728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:17.909012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:17.909310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:17.919219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:55:17.927788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:55:17.928908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:55:17.929921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 592/1188] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:55:17.935104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1898a-7c44-4ec9-8451-4760c932747a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:18.038332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1898a-7c44-4ec9-8451-4760c932747a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:18.070733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:18.071386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:18.071666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:18.071939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:18.072680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:18.072981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:18.074382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:18.082709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 597/1189] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:18.088561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:18.186558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df246c-e70d-41ea-b386-92e1ab45e4a2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:18.219627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:18.220490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:18.220773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:18.221047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] PATH_INFO: `/v3/users/29eb89dc7369434d9b88df75a82c22b6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:18.222427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] RBAC: Authorizing `identity:get_user(user_id=29eb89dc7369434d9b88df75a82c22b6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:18.224038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:18.225675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 593/1190] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:55:18 2022] GET /identity/v3/users/29eb89dc7369434d9b88df75a82c22b6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:18.231416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:18.281792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:18.332367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cd10d8-2261-4512-bb49-f6e69435c8c7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:18.365290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515bbea2-31a5-4468-b54e-15392460033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:18.366159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbea2-31a5-4468-b54e-15392460033e None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:18.366443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbea2-31a5-4468-b54e-15392460033e None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:18.366721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515bbea2-31a5-4468-b54e-15392460033e None admin] PATH_INFO: `/v3/projects/29143af1cad442009432ec483e4d60e0` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:18.367895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bbea2-31a5-4468-b54e-15392460033e None admin] RBAC: Authorizing `identity:get_project(project_id=29143af1cad442009432ec483e4d60e0)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:18.369364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515bbea2-31a5-4468-b54e-15392460033e None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:18.371081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 598/1191] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:55:18 2022] GET /identity/v3/projects/29143af1cad442009432ec483e4d60e0 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:18.376968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:18.419812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:18.475120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a6eaec-fd9e-460e-92e1-d610404cf909 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:18.506990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1f5e2-7717-497b-8812-869f0b2a151d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:18.507861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1f5e2-7717-497b-8812-869f0b2a151d None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:18.508138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1f5e2-7717-497b-8812-869f0b2a151d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:18.508409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1f5e2-7717-497b-8812-869f0b2a151d None admin] PATH_INFO: `/v3/projects/29143af1cad442009432ec483e4d60e0/users/29eb89dc7369434d9b88df75a82c22b6/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:18.510332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1f5e2-7717-497b-8812-869f0b2a151d None admin] RBAC: Authorizing `identity:create_grant(project_id=29143af1cad442009432ec483e4d60e0, user_id=29eb89dc7369434d9b88df75a82c22b6, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:18.511745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1f5e2-7717-497b-8812-869f0b2a151d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:18.622107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 594/1192] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:55:18 2022] PUT /identity/v3/projects/29143af1cad442009432ec483e4d60e0/users/29eb89dc7369434d9b88df75a82c22b6/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:55:20.769675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8fc38-bf3c-4e5b-9054-3ca67944bc9e None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:20.770211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8fc38-bf3c-4e5b-9054-3ca67944bc9e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:20.770302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe8fc38-bf3c-4e5b-9054-3ca67944bc9e None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:20.771880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 599/1193] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:55:20.777035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eee4e-f3a6-4e4e-9d1c-1888f352a638 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:20.777459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eee4e-f3a6-4e4e-9d1c-1888f352a638 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:20.777623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eee4e-f3a6-4e4e-9d1c-1888f352a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:20.794709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9a3eee4e-f3a6-4e4e-9d1c-1888f352a638 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:55:20.795885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9a3eee4e-f3a6-4e4e-9d1c-1888f352a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:55:20.860204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 595/1194] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:55:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:55:20.883801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8fc38-bf3c-4e5b-9054-3ca67944bc9e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:20.982890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe8fc38-bf3c-4e5b-9054-3ca67944bc9e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:21.000167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:21.020106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:21.020941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:21.021219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:21.021897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:21.031577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:55:21.039917 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:55:21.041024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:55:21.041974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 600/1195] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:55:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:55:21.047077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eee4e-f3a6-4e4e-9d1c-1888f352a638 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:21.076799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:21.142537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eee4e-f3a6-4e4e-9d1c-1888f352a638 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:21.177921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:21.178581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:21.178859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:21.179129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:21.179869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:21.180172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:21.181559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:21.189317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 596/1196] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:55:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:21.195953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:21.296393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7b6da-1e80-447d-90ae-394fb09ace78 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:21.330416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:21.331280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:21.331594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:21.331981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] PATH_INFO: `/v3/users/29eb89dc7369434d9b88df75a82c22b6` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:21.333246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] RBAC: Authorizing `identity:get_user(user_id=29eb89dc7369434d9b88df75a82c22b6)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:21.334712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:21.336336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 601/1197] 10.222.0.35 () {56 vars in 1177 bytes} [Tue May 24 09:55:21 2022] GET /identity/v3/users/29eb89dc7369434d9b88df75a82c22b6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:21.342107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:21.433428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:21.439813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242042d5-69e8-4444-99f5-5cf30cc8e319 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:21.472933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:21.473655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:21.473941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:21.474219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:21.539582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:21.541022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:21.629421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:21.636454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 597/1198] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:55:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:55:21.642383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:21.688099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:21.739137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4be1-5bf4-4242-b4a0-c7ecf88b6454 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:21.770902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e60ad-d6af-4ad1-a002-a614e4c85e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:21.772018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e60ad-d6af-4ad1-a002-a614e4c85e60 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:21.772300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e60ad-d6af-4ad1-a002-a614e4c85e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:21.772576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e60ad-d6af-4ad1-a002-a614e4c85e60 None admin] PATH_INFO: `/v3/projects/29143af1cad442009432ec483e4d60e0` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:21.773755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e60ad-d6af-4ad1-a002-a614e4c85e60 None admin] RBAC: Authorizing `identity:get_project(project_id=29143af1cad442009432ec483e4d60e0)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:21.775196 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e60ad-d6af-4ad1-a002-a614e4c85e60 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:21.776863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 602/1199] 10.222.0.35 () {56 vars in 1186 bytes} [Tue May 24 09:55:21 2022] GET /identity/v3/projects/29143af1cad442009432ec483e4d60e0 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:21.783031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:55:21.878310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dff802a-2fe3-422c-ab75-deeb7cbdcd67 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:55:21.911492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52912670-8242-4d1d-8c9b-cf89e39c6066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:55:21.912066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52912670-8242-4d1d-8c9b-cf89e39c6066 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:55:21.912348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52912670-8242-4d1d-8c9b-cf89e39c6066 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:55:21.912621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52912670-8242-4d1d-8c9b-cf89e39c6066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:55:21.913464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52912670-8242-4d1d-8c9b-cf89e39c6066 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=29143af1cad442009432ec483e4d60e0, user.id=29eb89dc7369434d9b88df75a82c22b6) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:55:21.913773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52912670-8242-4d1d-8c9b-cf89e39c6066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:55:21.915186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52912670-8242-4d1d-8c9b-cf89e39c6066 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:55:21.920790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 598/1200] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:55:21 2022] GET /identity/v3/role_assignments?user.id=29eb89dc7369434d9b88df75a82c22b6&scope.project.id=29143af1cad442009432ec483e4d60e0&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:55:31.766685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=43 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:55:31.918282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=71 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:42.971502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4973734-8a19-4fa3-866e-eb9df57ad887 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:42.972107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4973734-8a19-4fa3-866e-eb9df57ad887 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:42.972107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4973734-8a19-4fa3-866e-eb9df57ad887 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:42.973463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 603/1201] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:56:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:56:42.978625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874989-e8f0-4602-b7ba-0500edb56832 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:42.979128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874989-e8f0-4602-b7ba-0500edb56832 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:42.979235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba874989-e8f0-4602-b7ba-0500edb56832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:42.996775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ba874989-e8f0-4602-b7ba-0500edb56832 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:56:42.997982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ba874989-e8f0-4602-b7ba-0500edb56832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:56:43.064770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 599/1202] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:56:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:56:43.089824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4973734-8a19-4fa3-866e-eb9df57ad887 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:43.191932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4973734-8a19-4fa3-866e-eb9df57ad887 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:43.246178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:43.246178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:43.246178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:43.246178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:43.246689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:43.246689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:43.254958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:43.256095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:43.257408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:43.257408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:43.257408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:43.257408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:43.257408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 604/1203] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:43.262143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874989-e8f0-4602-b7ba-0500edb56832 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:43.357166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba874989-e8f0-4602-b7ba-0500edb56832 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:43.385484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:43.391167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ed038-44bf-460e-9c1b-c7a65e8a018d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:43.391818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ed038-44bf-460e-9c1b-c7a65e8a018d None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:43.392095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ed038-44bf-460e-9c1b-c7a65e8a018d None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:43.392366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ed038-44bf-460e-9c1b-c7a65e8a018d None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:43.393079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ed038-44bf-460e-9c1b-c7a65e8a018d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:43.393422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ed038-44bf-460e-9c1b-c7a65e8a018d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:43.394784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ed038-44bf-460e-9c1b-c7a65e8a018d None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:43.406303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 600/1204] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:43.412102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:43.463776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:43.508436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeed238-361e-4822-8957-0bd79fc793f3 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:43.541340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84878da3-7446-4825-891f-fc6c93364dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:43.541995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878da3-7446-4825-891f-fc6c93364dc8 None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:43.542351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878da3-7446-4825-891f-fc6c93364dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:43.542730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84878da3-7446-4825-891f-fc6c93364dc8 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:43.543452 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84878da3-7446-4825-891f-fc6c93364dc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:43.544905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84878da3-7446-4825-891f-fc6c93364dc8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:43.706777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 605/1205] 10.222.0.35 () {60 vars in 1132 bytes} [Tue May 24 09:56:43 2022] POST /identity/v3/users => generated 251 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:56:45.952583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b7f5b-2832-4c60-81aa-2022a7efa1a4 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:45.953161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b7f5b-2832-4c60-81aa-2022a7efa1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:45.953161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b7f5b-2832-4c60-81aa-2022a7efa1a4 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:45.954541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 601/1206] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:56:45.959912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea2c9f-3f5b-443d-9e55-649d43470c1b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:45.960295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea2c9f-3f5b-443d-9e55-649d43470c1b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:45.960463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea2c9f-3f5b-443d-9e55-649d43470c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:45.977565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4bea2c9f-3f5b-443d-9e55-649d43470c1b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:56:45.978747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4bea2c9f-3f5b-443d-9e55-649d43470c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:56:46.044830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 606/1207] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:56:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:56:46.068049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b7f5b-2832-4c60-81aa-2022a7efa1a4 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:46.174469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b7f5b-2832-4c60-81aa-2022a7efa1a4 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:46.208175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:46.208946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:46.209232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:46.209542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:46.219901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:56:46.227652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:46.228861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:46.230389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 602/1208] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:56:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:46.235597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea2c9f-3f5b-443d-9e55-649d43470c1b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:46.274929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:46.333104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea2c9f-3f5b-443d-9e55-649d43470c1b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:46.365790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62651419-6e5a-42be-a34f-da841995211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:46.366466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62651419-6e5a-42be-a34f-da841995211a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:46.366855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62651419-6e5a-42be-a34f-da841995211a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:46.367131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62651419-6e5a-42be-a34f-da841995211a None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:46.367877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62651419-6e5a-42be-a34f-da841995211a None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:46.368184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62651419-6e5a-42be-a34f-da841995211a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:46.369575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62651419-6e5a-42be-a34f-da841995211a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:46.377805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 607/1209] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:56:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:46.383641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:46.434357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:46.484614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe9ca9-d6b3-4aec-b5a8-2b45444a8418 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:46.517479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:46.518264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:46.518547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:46.518824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:46.530123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:46.531631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:46.550006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:46.550991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:46.551904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:46.551904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:46.551904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:46.551904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:46.551904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 603/1210] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:46.557001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62651419-6e5a-42be-a34f-da841995211a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:46.652376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:46.655156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62651419-6e5a-42be-a34f-da841995211a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:46.687141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:46.687797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:46.688075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:46.688346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:46.689064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:46.689394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:46.690763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:46.701434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 608/1211] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:46.707019 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:46.803595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b334f-aaa1-4d25-a7b0-b9f10511a9ca None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:46.805028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:46.836375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:46.837160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:46.837480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:46.837758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:46.845290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:46.846718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:56:46.860921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:46.862145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:46.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 604/1212] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:46.868038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:46.969207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a09cb-92f6-4d6f-914b-90e0f9fae7c6 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:47.001779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:47.002441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:47.002719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:47.002994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:47.003824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:47.004124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:47.005572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:47.022865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 609/1213] 10.222.0.35 () {56 vars in 1143 bytes} [Tue May 24 09:56:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:47.029486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:47.126608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba967e6a-18fe-440c-8c1c-9ecf61870386 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:47.160559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:47.161197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:47.161505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:47.161780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:47.191512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:47.227736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:47.229290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:47.322842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 605/1214] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:56:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:56:47.328999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:47.347880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:47.424887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed3fab5-017c-451d-9fd5-8fd297af02e1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:47.458123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:47.458897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:47.459174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:47.459451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:47.470214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:47.471844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:47.489909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:47.491432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:47.493470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:47.493470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:47.493470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:47.493470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:47.493470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 610/1215] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:47.498301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:47.557522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:47.595284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e02b100-b5c4-45e1-84cb-5db062a62563 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:47.627449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:47.628101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:47.628381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:47.628654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:47.629464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:47.629761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:47.631115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:47.642916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 606/1216] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:47.648799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:47.718143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:47.747656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78974866-ee0e-4b7a-aa0c-34d8e3c2ead9 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:47.783365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:47.784150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:47.784435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:47.784712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:47.791444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:47.792995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:56:47.806406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:47.807541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:47.809151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 611/1217] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:56:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:47.813627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:47.914415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7b4e7-eac6-4cf5-afbe-49e6db347a17 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:47.938270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:47.950435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f57446-b583-41f9-a7ce-f578b23d9d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:47.951091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57446-b583-41f9-a7ce-f578b23d9d92 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:47.951378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57446-b583-41f9-a7ce-f578b23d9d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:47.951658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57446-b583-41f9-a7ce-f578b23d9d92 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:47.952445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f57446-b583-41f9-a7ce-f578b23d9d92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:47.952758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f57446-b583-41f9-a7ce-f578b23d9d92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:47.954338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f57446-b583-41f9-a7ce-f578b23d9d92 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:47.966119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 607/1218] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:56:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:47.972382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:48.069449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd206be-4aba-4981-8db5-6ea91b256b6b None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:48.096890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:48.102478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ec952-c049-4335-9ecf-1d17419ec85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:48.103120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ec952-c049-4335-9ecf-1d17419ec85d None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:48.103398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ec952-c049-4335-9ecf-1d17419ec85d None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:48.103668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ec952-c049-4335-9ecf-1d17419ec85d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:48.104516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ec952-c049-4335-9ecf-1d17419ec85d None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=4989f980a8804f18b7ea2fec0ecad8a7) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:48.104814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ec952-c049-4335-9ecf-1d17419ec85d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:48.106443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74ec952-c049-4335-9ecf-1d17419ec85d None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:48.111636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 612/1219] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:56:47 2022] GET /identity/v3/role_assignments?user.id=4989f980a8804f18b7ea2fec0ecad8a7&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:50.289670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815648b3-d719-4453-9d8a-f8744fe3a06e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:50.290193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815648b3-d719-4453-9d8a-f8744fe3a06e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:50.290285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815648b3-d719-4453-9d8a-f8744fe3a06e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:50.291625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 608/1220] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:56:50.297502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b820d5d-655a-4aaa-9ea6-c7a8b3e4f325 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:50.297945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b820d5d-655a-4aaa-9ea6-c7a8b3e4f325 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:50.298040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b820d5d-655a-4aaa-9ea6-c7a8b3e4f325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:50.314477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4b820d5d-655a-4aaa-9ea6-c7a8b3e4f325 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:56:50.315637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4b820d5d-655a-4aaa-9ea6-c7a8b3e4f325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:56:50.379483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 613/1221] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:56:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:56:50.403835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815648b3-d719-4453-9d8a-f8744fe3a06e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:50.505523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815648b3-d719-4453-9d8a-f8744fe3a06e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:50.540188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:50.540980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:50.541307 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:50.541593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:50.552890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:56:50.561728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:50.562845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:50.563835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 609/1222] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:56:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:50.569229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b820d5d-655a-4aaa-9ea6-c7a8b3e4f325 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:50.665861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b820d5d-655a-4aaa-9ea6-c7a8b3e4f325 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:50.689530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:50.700139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:50.700802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:50.701111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:50.701432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:50.702182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] RBAC: Adding query filter params (name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:50.702486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:50.703861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:50.711567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 614/1223] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:56:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:56:50.717604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:50.763007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:50.818597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440447f-93e7-48df-a0a6-63bfe31446b9 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:50.850083 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:50.850868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:50.851146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:50.851419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:50.862287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:50.867002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:50.880589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:50.881665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:50.882854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:50.882854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:50.882854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:50.882854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:50.882854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 610/1224] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:50.887628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:50.985036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2df4-05b6-4bc3-b5eb-dbd9b91b5784 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:51.019833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:51.020415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:51.020693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:51.020967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:51.021796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:51.022103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:51.023731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:51.037811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 615/1225] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:51.044918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:51.144041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:51.150839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71cccd-1803-4f96-898d-17c8a76b35a2 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:51.185868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:51.186666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:51.186946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:51.187220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:51.194367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:51.195802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:56:51.210555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:51.211675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:51.212871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 611/1226] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:51.218464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:51.264373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:51.314138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda8999-a8f0-42f9-b247-234fe7ea3385 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:51.347869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:51.348541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:51.348822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:51.349097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:51.349958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:51.350266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:51.351698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:51.369601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 616/1227] 10.222.0.35 () {56 vars in 1143 bytes} [Tue May 24 09:56:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:51.375493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:51.475360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b588ef-59e0-44eb-a3d9-4bb86e79c1d5 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:51.507877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:51.508660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:51.508939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:51.509209 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:51.519523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:51.520952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:51.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:51.539848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:51.540999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:51.540999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:51.540999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:51.540999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:51.540999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 612/1228] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:51.545906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:51.644233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144862e-cc24-429c-8ca7-6c6ce29b414a None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:51.646496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:51.678209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:51.678858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:51.679137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:51.679406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:51.680120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:51.680420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:51.681809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:51.693092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 617/1229] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:51.698934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:51.707627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:51.794401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf2b99-e484-455f-adbb-0e4882885719 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:51.826042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:51.826822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:51.827101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:51.827373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:51.834148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:51.835575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:56:51.849638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:51.857050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:51.858201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 613/1230] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:56:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:51.858795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:51.954979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09808a-d0dd-4a7f-a3cf-0c5860ab6a91 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:51.988170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562412f-38c4-472e-bc90-0d0d430692f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:51.988818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562412f-38c4-472e-bc90-0d0d430692f9 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:51.989099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562412f-38c4-472e-bc90-0d0d430692f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:51.989401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562412f-38c4-472e-bc90-0d0d430692f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:51.990179 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562412f-38c4-472e-bc90-0d0d430692f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:51.990483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562412f-38c4-472e-bc90-0d0d430692f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:51.991902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562412f-38c4-472e-bc90-0d0d430692f9 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:52.004086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 618/1231] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:56:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:52.010183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:52.078020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:52.111743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115cbc4b-0b34-419c-a4a1-51b2451c4244 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:52.144279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f797dd-3fce-4076-af3b-4671e8017c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:52.145158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f797dd-3fce-4076-af3b-4671e8017c19 None admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:52.145465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f797dd-3fce-4076-af3b-4671e8017c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:52.145739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f797dd-3fce-4076-af3b-4671e8017c19 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/4989f980a8804f18b7ea2fec0ecad8a7/roles/36d82dc24b2b44ee935e79f13d0f521c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:52.160980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f797dd-3fce-4076-af3b-4671e8017c19 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=4989f980a8804f18b7ea2fec0ecad8a7, role_id=36d82dc24b2b44ee935e79f13d0f521c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:52.162566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f797dd-3fce-4076-af3b-4671e8017c19 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:52.298365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 614/1232] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:56:52 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/4989f980a8804f18b7ea2fec0ecad8a7/roles/36d82dc24b2b44ee935e79f13d0f521c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 09:56:54.473791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7fda9-87d2-4a85-8cd8-aa78ef172dc3 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:54.474324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7fda9-87d2-4a85-8cd8-aa78ef172dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:54.474324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d7fda9-87d2-4a85-8cd8-aa78ef172dc3 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:54.475716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 619/1233] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:56:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:56:54.481258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a8ec2-2840-42f0-8946-d02fc4b43edf None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:54.481721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a8ec2-2840-42f0-8946-d02fc4b43edf None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:54.481854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a8ec2-2840-42f0-8946-d02fc4b43edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:54.498436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9f4a8ec2-2840-42f0-8946-d02fc4b43edf None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:56:54.499620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9f4a8ec2-2840-42f0-8946-d02fc4b43edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:56:54.567060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 615/1234] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:56:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 24 09:56:54.591567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7fda9-87d2-4a85-8cd8-aa78ef172dc3 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:54.617040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:54.692819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d7fda9-87d2-4a85-8cd8-aa78ef172dc3 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:54.726459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:54.727189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:54.727467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:54.727736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:54.737536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:56:54.744996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:54.746661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:54.747675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 620/1235] 10.222.0.35 () {56 vars in 1102 bytes} [Tue May 24 09:56:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:54.752914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a8ec2-2840-42f0-8946-d02fc4b43edf None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:54.798683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:54.850575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a8ec2-2840-42f0-8946-d02fc4b43edf None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:54.884161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:54.884825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:54.885110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:54.885418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:54.886163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] RBAC: Adding query filter params (name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:54.887149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:54.888529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:54.896346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 616/1236] 10.222.0.35 () {56 vars in 1103 bytes} [Tue May 24 09:56:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:54.902440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:54.985813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:55.001287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c3a63-fede-4f9e-87f7-84585164a76f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:55.033811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:55.034599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:55.034879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:55.035153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:55.046045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:55.047513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:55.065643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:55.066835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:55.067955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:55.067955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:55.067955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:55.067955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:55.067955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 621/1237] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:55.072650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:55.171191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0e5fc-48e8-402a-b8b8-5bb1046a205f None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:55.179969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:55.205034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:55.205718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:55.206004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:55.206284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:55.207006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:55.207316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:55.208680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:55.220204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 617/1238] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:55.226444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:55.326950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ac9bc0-8b6a-4e15-94b9-40e5193e97f1 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:55.359400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:55.360196 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:55.360477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:55.360750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:55.363518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:55.368776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:55.370262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:56:55.384688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:55.385841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:55.387074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 622/1239] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:55.391933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:55.490866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5dbfd-fd4a-44ed-9cb4-8c95b21156e1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:55.524144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:55.524800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:55.525078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:55.525374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:55.526198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:55.526502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:55.528569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:55.546773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 618/1240] 10.222.0.35 () {56 vars in 1143 bytes} [Tue May 24 09:56:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:55.552678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:55.649840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d0a1e-1368-46a2-a68b-da431f0711d5 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:55.682900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:55.683539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:55.683818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:55.684091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:55.742806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:55.749304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:55.751198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:55.844908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 623/1241] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:56:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:56:55.850724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:55.877665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:55.948507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080041a-fa8c-4f1f-bf65-602a4da8528c None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:55.984015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:55.984841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:55.985121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:55.985425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:55.995079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:55.997151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:56.014115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:56.016148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:56.017131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:56.017131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:56.017131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:56.017131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:56.017131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 619/1242] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:56.021316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:56.111682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:56.117310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddb6a6-2467-4f1e-9456-bab6a7ba522c None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:56.150448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:56.151113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:56.151390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:56.151659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:56.153044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:56.153379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:56.154742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:56.165836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 624/1243] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:56.172156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:56.247753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:56.267790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f5cf9-aa36-4326-ad5a-e3ceaec3e4b3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:56.300348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:56.301124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:56.301497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:56.301776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:56.308063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:56.309617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:56:56.322839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:56.323944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:56:56.325408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 620/1244] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:56:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:56.329957 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:56.428209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd19e2-1333-4a90-9474-247d0fd752d8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:56.460401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689a3850-21b5-486e-a940-92642f3d1ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:56.461056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a3850-21b5-486e-a940-92642f3d1ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:56.461477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a3850-21b5-486e-a940-92642f3d1ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:56.461752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a3850-21b5-486e-a940-92642f3d1ec2 None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:56.462535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a3850-21b5-486e-a940-92642f3d1ec2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:56.462841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a3850-21b5-486e-a940-92642f3d1ec2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:56.464264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689a3850-21b5-486e-a940-92642f3d1ec2 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:56.476927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 625/1245] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:56:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:56.483916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:56.582226 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968da99f-4986-481a-8843-8db0c5d8d489 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:56.615969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ea91d-1050-4e0e-b050-58a711053a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:56.616615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ea91d-1050-4e0e-b050-58a711053a04 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:56.616894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ea91d-1050-4e0e-b050-58a711053a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:56.617166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ea91d-1050-4e0e-b050-58a711053a04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:56.618009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ea91d-1050-4e0e-b050-58a711053a04 None admin] RBAC: Adding query filter params (role.id=36d82dc24b2b44ee935e79f13d0f521c, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=4989f980a8804f18b7ea2fec0ecad8a7) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:56.618316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ea91d-1050-4e0e-b050-58a711053a04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:56.619854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ea91d-1050-4e0e-b050-58a711053a04 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:56.625661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 621/1246] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:56:56 2022] GET /identity/v3/role_assignments?user.id=4989f980a8804f18b7ea2fec0ecad8a7&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=36d82dc24b2b44ee935e79f13d0f521c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:58.909548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a0a53-2e46-4939-8e58-6787a985c627 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:58.910084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a0a53-2e46-4939-8e58-6787a985c627 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:58.910084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a0a53-2e46-4939-8e58-6787a985c627 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:58.911511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 626/1247] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:56:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:56:58.916645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462598a-7c68-4494-99bb-ee984cae329a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:58.917039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462598a-7c68-4494-99bb-ee984cae329a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:58.917202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462598a-7c68-4494-99bb-ee984cae329a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:58.922370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:58.934452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a462598a-7c68-4494-99bb-ee984cae329a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:56:58.935681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a462598a-7c68-4494-99bb-ee984cae329a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:56:58.999971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 622/1248] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:56:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:56:59.023817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a0a53-2e46-4939-8e58-6787a985c627 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:59.042175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:59.122037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a0a53-2e46-4939-8e58-6787a985c627 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:59.157377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:59.158181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:59.158462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:59.158737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:59.169060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:56:59.176630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:59.177801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:59.178848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 627/1249] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:56:59.186533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462598a-7c68-4494-99bb-ee984cae329a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:59.283854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462598a-7c68-4494-99bb-ee984cae329a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:59.313016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:59.317820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:59.318484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:59.318769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:59.319045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:59.319819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:59.320124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:59.321514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:59.330351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 623/1250] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:56:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:59.336027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:59.413759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:59.436707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a39619-e1b9-43bc-ac5e-a0540d40a107 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:59.474272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:59.475062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:59.475342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:59.475611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:59.486152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:59.487589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:56:59.504425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:59.505623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:59.506934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:56:59.506934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:56:59.506934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:56:59.506934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:59.506934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 628/1251] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 24 09:56:59.511968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:59.612171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05035a-0eab-47b0-aa23-f2ed6022adc7 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:59.644806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:59.645477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:59.645764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:59.646040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:59.646785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:59.647092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:59.648705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:59.660436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 624/1252] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:56:59.666155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:59.765646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbc7f6-5109-470a-8e8f-df0f97087c67 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:59.791407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:59.799516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:59.800319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:59.800597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:59.800945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:59.808977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:59.810439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:56:59.824660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:56:59.825848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:56:59.827051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 629/1253] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:56:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:56:59.831773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:56:59.887915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:56:59.931549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb1021b-e465-4543-8c38-14f38d31d132 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:56:59.965239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:56:59.965941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:56:59.966224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:56:59.966501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:56:59.967331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:56:59.967641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:56:59.969062 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:56:59.986652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 625/1254] 10.222.0.35 () {56 vars in 1143 bytes} [Tue May 24 09:56:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:56:59.992562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:00.093350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c68e9-bfc5-4f93-a501-9c45301f7133 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:00.126315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:00.126954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:00.127237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:00.127514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:00.193436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:00.204825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:00.206337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:00.301328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 630/1255] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:56:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:57:00.307058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:00.406839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f83f76a-428e-47c8-95c6-223fa49ee3d1 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:00.419908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:00.439810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:00.440609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:00.440877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:00.441167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:00.456355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:00.456355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:57:00.472539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:00.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:00.479419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:57:00.479419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:57:00.479419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:00.479419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:00.479419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 626/1256] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:00.479703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:00.571563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:00.583678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf161d-c2d2-4b40-84c2-1a10d739324f None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:00.617522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:00.618179 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:00.618457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:00.618792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:00.619528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:00.620032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:00.622479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:00.633664 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 631/1257] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:00.639914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:00.738129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e9d467-6e14-496f-8529-b9c92dfef9d7 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:00.771800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:00.772584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:00.772869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:00.773150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:00.780089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:00.781568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:57:00.796278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:00.798030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:00.799553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 627/1258] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:00.804176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:00.902934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f077c0-2817-47a8-80ba-f34bf5bf6cba None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:00.938853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b6cfd4-53bf-4cdb-8c71-a793009cee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:00.939510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6cfd4-53bf-4cdb-8c71-a793009cee8c None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:00.939786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6cfd4-53bf-4cdb-8c71-a793009cee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:00.940057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b6cfd4-53bf-4cdb-8c71-a793009cee8c None admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:00.940834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6cfd4-53bf-4cdb-8c71-a793009cee8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:00.941139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6cfd4-53bf-4cdb-8c71-a793009cee8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:00.942622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b6cfd4-53bf-4cdb-8c71-a793009cee8c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:00.954790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 632/1259] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:57:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:00.960863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:00.986424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:01.063048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1c9703-d827-4a42-add7-6eb429454546 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:01.097883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab11273-bb15-4343-b3ac-585ef74b8d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:01.098606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab11273-bb15-4343-b3ac-585ef74b8d81 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:01.098885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab11273-bb15-4343-b3ac-585ef74b8d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:01.099182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab11273-bb15-4343-b3ac-585ef74b8d81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:01.100004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab11273-bb15-4343-b3ac-585ef74b8d81 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=4989f980a8804f18b7ea2fec0ecad8a7) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:01.100375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab11273-bb15-4343-b3ac-585ef74b8d81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:01.102139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab11273-bb15-4343-b3ac-585ef74b8d81 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:01.108717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 628/1260] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:57:00 2022] GET /identity/v3/role_assignments?user.id=4989f980a8804f18b7ea2fec0ecad8a7&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e1e5746208514068acadbab89907c473 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:03.274520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d8247-4572-4a95-a9df-b06559efb287 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:03.275046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d8247-4572-4a95-a9df-b06559efb287 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:03.275133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d8247-4572-4a95-a9df-b06559efb287 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:03.276482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 633/1261] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:57:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:57:03.282744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9aa8-e5d8-4928-b007-fa522c1b8822 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:03.283182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9aa8-e5d8-4928-b007-fa522c1b8822 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:03.283286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445f9aa8-e5d8-4928-b007-fa522c1b8822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:03.300476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-445f9aa8-e5d8-4928-b007-fa522c1b8822 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:57:03.301685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-445f9aa8-e5d8-4928-b007-fa522c1b8822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:57:03.366949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 629/1262] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:57:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:57:03.390120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d8247-4572-4a95-a9df-b06559efb287 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:03.401024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:03.489213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d8247-4572-4a95-a9df-b06559efb287 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:03.521563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:03.522343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:03.522621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:03.522895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:03.535508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:57:03.543259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:03.544404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:57:03.545474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 634/1263] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:57:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:03.550759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f9aa8-e5d8-4928-b007-fa522c1b8822 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:03.569090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:03.650381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445f9aa8-e5d8-4928-b007-fa522c1b8822 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:03.682629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:03.683808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:03.684083 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:03.684354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:03.685084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:03.685413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:03.686804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:03.694578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 630/1264] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:57:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:03.701327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:03.767834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:03.802538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d020cd-0fed-4963-bffe-9760b5df61db None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:03.837103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:03.837961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:03.845924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:03.845924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:03.849535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:03.851049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:57:03.868361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:03.869463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:03.874576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:57:03.874576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:57:03.874576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:03.874576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:57:03.874576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 635/1265] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:57:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:03.876222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:03.958378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:03.976775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a955a-14ec-40e1-9156-2955a17fd43e None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:04.010379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:04.011611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:04.011915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:04.012207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:04.012941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:04.013251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:04.014678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:04.026374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 631/1266] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:04.032262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:04.135681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad73eb1-1b38-4383-b00c-6cdbc20570ed None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:04.156464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:04.171689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:04.172536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:04.172863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:04.173176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:04.180732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:04.182241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:57:04.197081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:04.199478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:57:04.200693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 636/1267] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:57:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:04.204517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:04.304818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8c216-043c-43b7-bfa7-2ec4daa39665 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:04.338787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:04.339591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:04.339871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:04.340146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:04.340974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:04.341308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:04.342720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:04.351905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:04.360538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 632/1268] 10.222.0.35 () {56 vars in 1143 bytes} [Tue May 24 09:57:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:04.366150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:04.468597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ff6088-1bf4-4cc4-803e-8cce016b4661 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:04.503876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:04.504714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:04.505040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:04.505388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:04.516187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:04.518108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:57:04.535787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:04.536739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:04.537729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:57:04.537729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:57:04.537729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:04.537729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:57:04.537729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 637/1269] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:04.544472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:04.643202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e441869-8d6b-42df-8885-b1c88505e021 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:04.678101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:04.678775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:04.679044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:04.679321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:04.680036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:04.680337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:04.681737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:04.692572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 633/1270] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:04.698303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:04.731816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:04.801020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387a256-9bce-4349-8760-a26f6a8fed75 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:04.834246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:04.835028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:04.835308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:04.835576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:04.843037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:04.844638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:57:04.861736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:04.865303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:57:04.866534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:57:04.867324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 638/1271] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:57:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:04.873434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:04.930983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:04.972667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200dc74-06d8-4a6c-b9de-6e7bc19f9d94 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:05.004775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0f772-25fe-4469-b8cf-133f7220a044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:05.005466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0f772-25fe-4469-b8cf-133f7220a044 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:05.005749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0f772-25fe-4469-b8cf-133f7220a044 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:05.006075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0f772-25fe-4469-b8cf-133f7220a044 None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:05.006861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0f772-25fe-4469-b8cf-133f7220a044 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:05.007180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0f772-25fe-4469-b8cf-133f7220a044 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:05.008607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0f772-25fe-4469-b8cf-133f7220a044 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:05.020645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 634/1272] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:57:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:05.027409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:05.118835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:05.127776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a49087-9aff-4741-95ae-e3bf53c290a8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:05.165738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf7e1b-acb4-4490-aab8-41732ab42769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:05.166627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7e1b-acb4-4490-aab8-41732ab42769 None admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:05.166908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7e1b-acb4-4490-aab8-41732ab42769 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:05.167252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf7e1b-acb4-4490-aab8-41732ab42769 None admin] PATH_INFO: `/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/4989f980a8804f18b7ea2fec0ecad8a7/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:05.169232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf7e1b-acb4-4490-aab8-41732ab42769 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bfce8a80cde4eadbdcc4c830e8c06fc, user_id=4989f980a8804f18b7ea2fec0ecad8a7, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:05.170737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf7e1b-acb4-4490-aab8-41732ab42769 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:05.254081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 639/1273] 10.222.0.35 () {58 vars in 1439 bytes} [Tue May 24 09:57:05 2022] PUT /identity/v3/projects/9bfce8a80cde4eadbdcc4c830e8c06fc/users/4989f980a8804f18b7ea2fec0ecad8a7/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 09:57:07.500347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44424d5-7d0e-4609-a34c-ae3d0c967e6a None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:07.500898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44424d5-7d0e-4609-a34c-ae3d0c967e6a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:07.500898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44424d5-7d0e-4609-a34c-ae3d0c967e6a None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:07.502285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 635/1274] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:57:07.507649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e215a46-2c96-464d-80e6-32250a33f49f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:07.508067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e215a46-2c96-464d-80e6-32250a33f49f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:07.508195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e215a46-2c96-464d-80e6-32250a33f49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:07.525118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4e215a46-2c96-464d-80e6-32250a33f49f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:57:07.526320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4e215a46-2c96-464d-80e6-32250a33f49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:57:07.592744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 640/1275] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:57:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:57:07.616203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44424d5-7d0e-4609-a34c-ae3d0c967e6a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:07.717017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44424d5-7d0e-4609-a34c-ae3d0c967e6a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:07.734911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:07.753155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:07.753155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:07.753155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:07.753155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:07.763474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(role_id) May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 24 09:57:07.771328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:07.772554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:07.773723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 636/1276] 10.222.0.35 () {56 vars in 1096 bytes} [Tue May 24 09:57:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:07.779548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e215a46-2c96-464d-80e6-32250a33f49f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:07.860652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:07.878704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e215a46-2c96-464d-80e6-32250a33f49f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:07.913684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:07.914351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:07.914631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:07.914949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:07.915695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:07.916000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:07.917620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:07.925580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 641/1277] 10.222.0.35 () {56 vars in 1099 bytes} [Tue May 24 09:57:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:07.931357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:08.031856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b457eb-810a-47e4-850a-333fbea7d7b4 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:08.064681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:08.065495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:08.065780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:08.066064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:08.077449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:08.078974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:57:08.096795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:08.097928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:08.098864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:57:08.098864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:57:08.098864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:08.098864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:08.098864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 637/1278] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:08.103804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:08.203779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d2416-5437-4b57-8fb9-559d8a39af06 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:08.235916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:08.236571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:08.236851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:08.237130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:08.237921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:08.238225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:08.239596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:08.252440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 642/1279] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:08.258188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:08.289461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:08.357553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289e59-2568-498f-8e67-a55d728f0157 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:08.393832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:08.394636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:08.394922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:08.395201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:08.402812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:08.404284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_user(user_id) May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 24 09:57:08.419387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:08.420449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:08.421494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 638/1280] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:57:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:08.426399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:08.430258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:08.525617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a902c34-13e6-422a-8a40-8beb7eca6453 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:08.562868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:08.562868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:08.562868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:08.562868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:08.563348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:08.563348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:08.563348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:08.580424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 643/1281] 10.222.0.35 () {56 vars in 1143 bytes} [Tue May 24 09:57:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:08.586245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:08.675485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:08.685338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ec107-c8dd-4c52-88c1-d49849820517 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:08.718011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:08.718650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:08.718932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:08.719211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:08.787261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:08.788738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:08.875155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:08.887832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 639/1282] 10.222.0.35 () {58 vars in 1282 bytes} [Tue May 24 09:57:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 24 09:57:08.893722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:08.954314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:08.992367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef04673-c703-47eb-967d-40ece5f7b644 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:09.026686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:09.027476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:09.027758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:09.028032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:09.038278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:09.039721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 24 09:57:09.057380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:09.059126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:09.060208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 24 09:57:09.060208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 24 09:57:09.060208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 24 09:57:09.060208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:57:09.060208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 644/1283] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 09:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:09.064899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:09.163908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452ab7a-9155-455d-834c-2a187cb7f03a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:09.199979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:09.200631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:09.200953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:09.201233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:09.201974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:09.202276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:09.203625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:09.215103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 640/1284] 10.222.0.35 () {56 vars in 1109 bytes} [Tue May 24 09:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:09.220750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:09.318017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c0de45-0564-44fc-aed9-3fdaf131b4b8 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:09.333287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:09.351946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:09.355790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:09.355790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:09.355790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:09.360304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:09.361813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(project_id) May 24 09:57:09.377329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:09.378402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 09:57:09.379369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 645/1285] 10.222.0.35 () {56 vars in 1111 bytes} [Tue May 24 09:57:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:09.384466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:09.435678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:09.485021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13683094-c6be-44a2-b3ac-9ef61ba7dad3 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:09.519653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52847ee9-5cd2-4332-9ef6-a011e0a1647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:09.520312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52847ee9-5cd2-4332-9ef6-a011e0a1647f None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:09.520598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52847ee9-5cd2-4332-9ef6-a011e0a1647f None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:09.520868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52847ee9-5cd2-4332-9ef6-a011e0a1647f None admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:09.521681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52847ee9-5cd2-4332-9ef6-a011e0a1647f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:09.521987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52847ee9-5cd2-4332-9ef6-a011e0a1647f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:09.523674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52847ee9-5cd2-4332-9ef6-a011e0a1647f None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:09.535948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 641/1286] 10.222.0.35 () {56 vars in 1148 bytes} [Tue May 24 09:57:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:09.542719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:09.639628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8768d-dd81-433d-a0a8-bec07d18b1c7 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:09.674410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e31d25-aed5-48d0-b0e4-040af9cad494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:09.675054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e31d25-aed5-48d0-b0e4-040af9cad494 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:09.675342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e31d25-aed5-48d0-b0e4-040af9cad494 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:09.675606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e31d25-aed5-48d0-b0e4-040af9cad494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:09.676432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e31d25-aed5-48d0-b0e4-040af9cad494 None admin] RBAC: Adding query filter params (role.id=e1e5746208514068acadbab89907c473, scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc, user.id=4989f980a8804f18b7ea2fec0ecad8a7) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:09.676740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e31d25-aed5-48d0-b0e4-040af9cad494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:09.678262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e31d25-aed5-48d0-b0e4-040af9cad494 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:09.684109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 646/1287] 10.222.0.35 () {56 vars in 1374 bytes} [Tue May 24 09:57:09 2022] GET /identity/v3/role_assignments?user.id=4989f980a8804f18b7ea2fec0ecad8a7&scope.project.id=9bfce8a80cde4eadbdcc4c830e8c06fc&role.id=e1e5746208514068acadbab89907c473 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:11.912332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5f12c-59d7-4853-98e5-76801b3c5900 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:11.912903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5f12c-59d7-4853-98e5-76801b3c5900 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:11.912903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5f12c-59d7-4853-98e5-76801b3c5900 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:11.914366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 642/1288] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:57:11.920014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191ea3-93e5-47ec-b964-884473544610 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:11.920411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191ea3-93e5-47ec-b964-884473544610 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:11.920564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59191ea3-93e5-47ec-b964-884473544610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:11.937625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-59191ea3-93e5-47ec-b964-884473544610 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:57:11.938793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-59191ea3-93e5-47ec-b964-884473544610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:57:11.970381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:12.006658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 647/1289] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:57:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:57:12.029657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5f12c-59d7-4853-98e5-76801b3c5900 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:12.132356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5f12c-59d7-4853-98e5-76801b3c5900 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:12.143848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:12.166285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:12.167207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:12.167470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:12.167743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:12.168395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:12.169799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:12.181701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:12.188308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:12.189519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 643/1290] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:57:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:12.190539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59191ea3-93e5-47ec-b964-884473544610 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:12.286814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59191ea3-93e5-47ec-b964-884473544610 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:12.320337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ead72-f3c7-4eff-84c9-19c7e44f21f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:12.320997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ead72-f3c7-4eff-84c9-19c7e44f21f7 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:12.321341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ead72-f3c7-4eff-84c9-19c7e44f21f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:12.321664 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ead72-f3c7-4eff-84c9-19c7e44f21f7 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:12.322376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ead72-f3c7-4eff-84c9-19c7e44f21f7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:12.322683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ead72-f3c7-4eff-84c9-19c7e44f21f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:12.324084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ead72-f3c7-4eff-84c9-19c7e44f21f7 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:12.328847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 648/1291] 10.222.0.35 () {56 vars in 1116 bytes} [Tue May 24 09:57:12 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:12.334789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:12.434508 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1aea1-20f5-4a4b-9282-f14c1159277a None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:12.467178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f619db-ffcf-4ab8-9cf3-14422e4db9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:12.467834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f619db-ffcf-4ab8-9cf3-14422e4db9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:12.468113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f619db-ffcf-4ab8-9cf3-14422e4db9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:12.468387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f619db-ffcf-4ab8-9cf3-14422e4db9e3 None admin] PATH_INFO: `/v3/services` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:12.469089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f619db-ffcf-4ab8-9cf3-14422e4db9e3 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:12.469424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f619db-ffcf-4ab8-9cf3-14422e4db9e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:12.470780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f619db-ffcf-4ab8-9cf3-14422e4db9e3 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:12.475127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 644/1292] 10.222.0.35 () {56 vars in 1116 bytes} [Tue May 24 09:57:12 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:57:14.657863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36923d0-d095-4a43-b0fb-e572a1d0c5e0 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:14.658394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36923d0-d095-4a43-b0fb-e572a1d0c5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:14.658394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36923d0-d095-4a43-b0fb-e572a1d0c5e0 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:14.659786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 649/1293] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:57:14.665111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2ab80-7561-4a0b-acaf-1ff146fc1d19 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:14.665620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2ab80-7561-4a0b-acaf-1ff146fc1d19 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:14.665726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d2ab80-7561-4a0b-acaf-1ff146fc1d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:14.682287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-52d2ab80-7561-4a0b-acaf-1ff146fc1d19 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:57:14.683484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-52d2ab80-7561-4a0b-acaf-1ff146fc1d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:57:14.712704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:14.748768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 645/1294] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:57:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:57:14.771562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36923d0-d095-4a43-b0fb-e572a1d0c5e0 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:14.802854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:14.872024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36923d0-d095-4a43-b0fb-e572a1d0c5e0 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:14.905859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a9abee-c3a6-4727-b2f6-9577ac0caf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:14.906523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9abee-c3a6-4727-b2f6-9577ac0caf2c None admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:14.906806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9abee-c3a6-4727-b2f6-9577ac0caf2c None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:14.907081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a9abee-c3a6-4727-b2f6-9577ac0caf2c None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:14.907843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9abee-c3a6-4727-b2f6-9577ac0caf2c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:14.909259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a9abee-c3a6-4727-b2f6-9577ac0caf2c None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:15.000283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 650/1295] 10.222.0.35 () {60 vars in 1141 bytes} [Tue May 24 09:57:14 2022] POST /identity/v3/services => generated 246 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 09:57:17.231486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c0eeb-2591-4396-a807-2c69b19dd5fe None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:17.231486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c0eeb-2591-4396-a807-2c69b19dd5fe None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:17.231486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c0eeb-2591-4396-a807-2c69b19dd5fe None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:17.232672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 646/1296] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:57:17.237874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c34b2-f9d9-43cf-a6a9-e765c00ef4cd None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:17.238293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c34b2-f9d9-43cf-a6a9-e765c00ef4cd None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:17.238436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c34b2-f9d9-43cf-a6a9-e765c00ef4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:17.255407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-d97c34b2-f9d9-43cf-a6a9-e765c00ef4cd None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:57:17.256527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d97c34b2-f9d9-43cf-a6a9-e765c00ef4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:57:17.329847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 651/1297] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:57:17 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:57:17.352368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c0eeb-2591-4396-a807-2c69b19dd5fe None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:17.453661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c0eeb-2591-4396-a807-2c69b19dd5fe None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:17.486296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:17.487080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:17.487362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:17.487634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:17.488285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:17.489680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:17.501817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:17.504262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:17.505327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 647/1298] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:57:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:17.509038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c34b2-f9d9-43cf-a6a9-e765c00ef4cd None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:17.528366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:17.605728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c34b2-f9d9-43cf-a6a9-e765c00ef4cd None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:17.638931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:17.639620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:17.639896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:17.640164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:17.640871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:17.641159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:17.642560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:17.647630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 652/1299] 10.222.0.35 () {56 vars in 1116 bytes} [Tue May 24 09:57:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:17.654098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:17.686129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:17.753099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662bb6e1-3d11-4393-a185-5f57e459ae05 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:17.785399 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791ba99-7320-4e5b-ae1f-de8b963390bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:17.786044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791ba99-7320-4e5b-ae1f-de8b963390bf None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:17.786324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791ba99-7320-4e5b-ae1f-de8b963390bf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:17.786598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791ba99-7320-4e5b-ae1f-de8b963390bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:17.787384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791ba99-7320-4e5b-ae1f-de8b963390bf None admin] RBAC: Adding query filter params (interface=public, service_id=a921f5845c9f4330bbe1dcfaec28c0fc, region_id=RegionOne) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:17.787693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791ba99-7320-4e5b-ae1f-de8b963390bf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:17.789053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791ba99-7320-4e5b-ae1f-de8b963390bf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:17.794504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 648/1300] 10.222.0.35 () {56 vars in 1251 bytes} [Tue May 24 09:57:17 2022] GET /identity/v3/endpoints?service_id=a921f5845c9f4330bbe1dcfaec28c0fc&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 09:57:17.800321 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:17.881166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:17.898485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98794d2c-3b8b-4770-8662-ddd5bfc84181 None admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:17.932039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e99ca-0a2c-489d-9e77-9192ad0bc550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:17.932705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e99ca-0a2c-489d-9e77-9192ad0bc550 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:17.933039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e99ca-0a2c-489d-9e77-9192ad0bc550 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:17.933380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e99ca-0a2c-489d-9e77-9192ad0bc550 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:17.934057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e99ca-0a2c-489d-9e77-9192ad0bc550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:17.935420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e99ca-0a2c-489d-9e77-9192ad0bc550 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:17.941973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 653/1301] 10.222.0.35 () {56 vars in 1087 bytes} [Tue May 24 09:57:17 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 09:57:20.175256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29e832-c7d5-4120-a858-6e9549b063ad None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:20.175692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29e832-c7d5-4120-a858-6e9549b063ad None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:20.175839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c29e832-c7d5-4120-a858-6e9549b063ad None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:20.177197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 649/1302] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 09:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 09:57:20.182432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bb6cb-2272-4ee8-ae30-457a56cc795e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:20.182809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bb6cb-2272-4ee8-ae30-457a56cc795e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:20.183062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bb6cb-2272-4ee8-ae30-457a56cc795e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:20.199477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a52bb6cb-2272-4ee8-ae30-457a56cc795e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 09:57:20.200698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a52bb6cb-2272-4ee8-ae30-457a56cc795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 09:57:20.266786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 654/1303] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 09:57:20 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 24 09:57:20.291586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29e832-c7d5-4120-a858-6e9549b063ad None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:20.388074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:20.394756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c29e832-c7d5-4120-a858-6e9549b063ad None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:20.429309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:20.430052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:20.430382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:20.430660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:20.431308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:20.432899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 09:57:20.444524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_service(service_id) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 09:57:20.445733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 09:57:20.446804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 650/1304] 10.222.0.35 () {56 vars in 1117 bytes} [Tue May 24 09:57:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 24 09:57:20.452298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bb6cb-2272-4ee8-ae30-457a56cc795e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:20.528274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:20.549588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bb6cb-2272-4ee8-ae30-457a56cc795e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:20.582011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b49b7-5e5b-4b88-88d7-8ed374b9add1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:20.582663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b49b7-5e5b-4b88-88d7-8ed374b9add1 None admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:20.582946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b49b7-5e5b-4b88-88d7-8ed374b9add1 None admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:20.583220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b49b7-5e5b-4b88-88d7-8ed374b9add1 None admin] PATH_INFO: `/v3/services` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:20.583923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b49b7-5e5b-4b88-88d7-8ed374b9add1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 09:57:20.584226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b49b7-5e5b-4b88-88d7-8ed374b9add1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:20.585622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b49b7-5e5b-4b88-88d7-8ed374b9add1 None admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:20.591243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 655/1305] 10.222.0.35 () {56 vars in 1116 bytes} [Tue May 24 09:57:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 09:57:20.597199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 09:57:20.698980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5069e3e-0205-42de-999a-0cef31b0a679 None admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 09:57:20.731495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea7f9e-280d-4946-9b04-c6602afbaecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 09:57:20.732136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7f9e-280d-4946-9b04-c6602afbaecf None admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 09:57:20.732415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7f9e-280d-4946-9b04-c6602afbaecf None admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 09:57:20.732687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea7f9e-280d-4946-9b04-c6602afbaecf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 09:57:20.733497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea7f9e-280d-4946-9b04-c6602afbaecf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 09:57:20.734911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea7f9e-280d-4946-9b04-c6602afbaecf None admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 09:57:20.825496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 651/1306] 10.222.0.35 () {60 vars in 1144 bytes} [Tue May 24 09:57:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 09:57:30.645426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=30 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 09:57:30.820295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:39.963539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc717c1-1369-4d7c-96b7-d0b025a5aaf2 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:39.964015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc717c1-1369-4d7c-96b7-d0b025a5aaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:39.964102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc717c1-1369-4d7c-96b7-d0b025a5aaf2 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:39.965801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 656/1307] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:39.971007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8c7ca-08dc-409a-980f-2131cd808ee7 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:39.971431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8c7ca-08dc-409a-980f-2131cd808ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:39.971652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8c7ca-08dc-409a-980f-2131cd808ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:40.062269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-3dd8c7ca-08dc-409a-980f-2131cd808ee7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:40.063468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3dd8c7ca-08dc-409a-980f-2131cd808ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:40.192340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 652/1308] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:04:40.242407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12991-f619-4666-bbb7-5fb909759ab2 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:40.242742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12991-f619-4666-bbb7-5fb909759ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:40.242993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e12991-f619-4666-bbb7-5fb909759ab2 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:40.244750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 657/1309] 10.222.0.35 () {54 vars in 964 bytes} [Tue May 24 10:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:40.250856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60993d-a7ea-43d3-be90-2a727cb04525 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:40.251077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60993d-a7ea-43d3-be90-2a727cb04525 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:40.251355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60993d-a7ea-43d3-be90-2a727cb04525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:40.334763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8e60993d-a7ea-43d3-be90-2a727cb04525 None None] MFA Rules not processed for user `9937f1088b304574935fcdf579e4de7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:40.335952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8e60993d-a7ea-43d3-be90-2a727cb04525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:40.358104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:40.443380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 653/1310] 10.222.0.35 () {58 vars in 1061 bytes} [Tue May 24 10:04:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:04:40.454667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e12991-f619-4666-bbb7-5fb909759ab2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:04:40.456611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d3e12991-f619-4666-bbb7-5fb909759ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:40.648737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e12991-f619-4666-bbb7-5fb909759ab2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:04:40.731058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905a7b69-46a6-4560-8f44-87dbc517c8f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:04:40.731717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a7b69-46a6-4560-8f44-87dbc517c8f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:40.731999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a7b69-46a6-4560-8f44-87dbc517c8f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:40.732271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a7b69-46a6-4560-8f44-87dbc517c8f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:40.734027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-905a7b69-46a6-4560-8f44-87dbc517c8f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:40.791688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:40.853841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a7b69-46a6-4560-8f44-87dbc517c8f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:04:40.855315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905a7b69-46a6-4560-8f44-87dbc517c8f3 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:04:40.877662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 658/1311] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:04:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:04:43.368679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f9654-910a-44d6-8ef2-17db7de958b9 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:43.373942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f9654-910a-44d6-8ef2-17db7de958b9 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:43.373942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f9654-910a-44d6-8ef2-17db7de958b9 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:43.373942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 654/1312] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:04:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:43.376620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09caa5f-94ba-4a9d-9fe4-2de9e8f80787 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:43.377078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09caa5f-94ba-4a9d-9fe4-2de9e8f80787 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:43.377169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09caa5f-94ba-4a9d-9fe4-2de9e8f80787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:43.463593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f09caa5f-94ba-4a9d-9fe4-2de9e8f80787 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:43.464857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f09caa5f-94ba-4a9d-9fe4-2de9e8f80787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:43.570639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 659/1313] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:04:43.609737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f9654-910a-44d6-8ef2-17db7de958b9 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:04:43.611283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-835f9654-910a-44d6-8ef2-17db7de958b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:43.721292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f9654-910a-44d6-8ef2-17db7de958b9 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:04:43.795186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:43.797681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd6c95-882d-40ec-86c8-6c75cd121aac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:04:43.798324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6c95-882d-40ec-86c8-6c75cd121aac service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:43.798604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6c95-882d-40ec-86c8-6c75cd121aac service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:43.798876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd6c95-882d-40ec-86c8-6c75cd121aac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:43.800489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a6fd6c95-882d-40ec-86c8-6c75cd121aac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:43.912665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd6c95-882d-40ec-86c8-6c75cd121aac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:04:43.914116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fd6c95-882d-40ec-86c8-6c75cd121aac service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:04:43.933588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 655/1314] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:04:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:04:46.717393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6556e-fdc9-41e8-a8c1-35934f4fc1dc None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:46.720821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6556e-fdc9-41e8-a8c1-35934f4fc1dc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:46.720821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e6556e-fdc9-41e8-a8c1-35934f4fc1dc None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:46.720821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 660/1315] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:04:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:46.724540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18b04-4667-476b-9a4d-c9e52ed71460 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:46.724954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18b04-4667-476b-9a4d-c9e52ed71460 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:46.725105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18b04-4667-476b-9a4d-c9e52ed71460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:46.790266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-54e18b04-4667-476b-9a4d-c9e52ed71460 None None] MFA Rules not processed for user `90d9e3351e21469a95b1a88347c81225`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:46.791464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-54e18b04-4667-476b-9a4d-c9e52ed71460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:46.843795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:46.888090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 656/1316] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:04:46 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:04:46.943629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d98e8-97c1-44af-a171-fb8c2d4c575c None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:46.943937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d98e8-97c1-44af-a171-fb8c2d4c575c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:46.944187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d98e8-97c1-44af-a171-fb8c2d4c575c None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:46.945656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 661/1317] 10.222.0.35 () {54 vars in 964 bytes} [Tue May 24 10:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:46.952322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497f91f-2ff7-477c-9f77-e076c7237e07 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:46.952614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497f91f-2ff7-477c-9f77-e076c7237e07 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:46.952893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497f91f-2ff7-477c-9f77-e076c7237e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:47.028226 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0497f91f-2ff7-477c-9f77-e076c7237e07 None None] MFA Rules not processed for user `9937f1088b304574935fcdf579e4de7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:47.029618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0497f91f-2ff7-477c-9f77-e076c7237e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:47.106310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 657/1318] 10.222.0.35 () {58 vars in 1061 bytes} [Tue May 24 10:04:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:04:47.115789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d98e8-97c1-44af-a171-fb8c2d4c575c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:04:47.117475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6e6d98e8-97c1-44af-a171-fb8c2d4c575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:47.127376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:47.223359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d98e8-97c1-44af-a171-fb8c2d4c575c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:04:47.295022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f2928-f855-462b-9f20-0b956f72b0d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:04:47.295583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f2928-f855-462b-9f20-0b956f72b0d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:47.295862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f2928-f855-462b-9f20-0b956f72b0d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:47.296133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f2928-f855-462b-9f20-0b956f72b0d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:47.297781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f71f2928-f855-462b-9f20-0b956f72b0d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:47.357360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:47.403269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f2928-f855-462b-9f20-0b956f72b0d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:04:47.404704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f2928-f855-462b-9f20-0b956f72b0d3 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:04:47.422797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 662/1319] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:04:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:04:50.781755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a136-c3ec-4129-9995-a518cc04249d None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:50.782331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a136-c3ec-4129-9995-a518cc04249d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:50.782331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91a136-c3ec-4129-9995-a518cc04249d None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:50.783651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 658/1320] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:50.788934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b432cf-f8f6-4eb4-876d-3af855ceb6fd None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:50.789367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b432cf-f8f6-4eb4-876d-3af855ceb6fd None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:50.789516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b432cf-f8f6-4eb4-876d-3af855ceb6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:50.853190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-72b432cf-f8f6-4eb4-876d-3af855ceb6fd None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:50.854415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-72b432cf-f8f6-4eb4-876d-3af855ceb6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:50.949219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 663/1321] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:04:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:04:50.986835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a136-c3ec-4129-9995-a518cc04249d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:04:50.988373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6a91a136-c3ec-4129-9995-a518cc04249d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:51.038428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:51.090885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91a136-c3ec-4129-9995-a518cc04249d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:04:51.161638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab648a9-1948-4de8-bf29-fcdb5cb38293 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:04:51.162279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab648a9-1948-4de8-bf29-fcdb5cb38293 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:51.162558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab648a9-1948-4de8-bf29-fcdb5cb38293 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:51.162827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab648a9-1948-4de8-bf29-fcdb5cb38293 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:51.164457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9ab648a9-1948-4de8-bf29-fcdb5cb38293 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:51.257546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:51.270098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab648a9-1948-4de8-bf29-fcdb5cb38293 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:04:51.271515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab648a9-1948-4de8-bf29-fcdb5cb38293 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:04:51.289213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 659/1322] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:04:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:04:54.537369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730bc3f-b538-452d-9707-0721d409612f None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:54.537933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730bc3f-b538-452d-9707-0721d409612f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:54.537933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730bc3f-b538-452d-9707-0721d409612f None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:54.539284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 664/1323] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:54.544875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589182f-0d9f-4e19-928a-dd2fb697a077 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:54.545336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589182f-0d9f-4e19-928a-dd2fb697a077 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:54.545548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589182f-0d9f-4e19-928a-dd2fb697a077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:54.614003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c589182f-0d9f-4e19-928a-dd2fb697a077 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:54.615208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c589182f-0d9f-4e19-928a-dd2fb697a077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:54.711141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 660/1324] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:04:54.747569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730bc3f-b538-452d-9707-0721d409612f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:04:54.749118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0730bc3f-b538-452d-9707-0721d409612f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:54.769089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:54.852075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730bc3f-b538-452d-9707-0721d409612f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:04:54.925745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b997b9b9-f462-4a7d-8350-ca017dd28043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:04:54.926400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997b9b9-f462-4a7d-8350-ca017dd28043 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:54.926680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997b9b9-f462-4a7d-8350-ca017dd28043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:54.926951 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b997b9b9-f462-4a7d-8350-ca017dd28043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:54.928579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b997b9b9-f462-4a7d-8350-ca017dd28043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:55.004450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:55.043758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997b9b9-f462-4a7d-8350-ca017dd28043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:04:55.045501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b997b9b9-f462-4a7d-8350-ca017dd28043 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:04:55.066312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 665/1325] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:04:57.339887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a00ce7-6c8c-4ba0-9d36-a63252b84c1b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:57.340585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a00ce7-6c8c-4ba0-9d36-a63252b84c1b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:57.340585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a00ce7-6c8c-4ba0-9d36-a63252b84c1b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:57.341958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 661/1326] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:04:57.347188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866ef33-f3d1-42f1-bbea-3e26e9535cfe None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:57.347600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866ef33-f3d1-42f1-bbea-3e26e9535cfe None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:57.347861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866ef33-f3d1-42f1-bbea-3e26e9535cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:57.412550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2866ef33-f3d1-42f1-bbea-3e26e9535cfe None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:04:57.413844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2866ef33-f3d1-42f1-bbea-3e26e9535cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:57.513594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 666/1327] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:04:57.550984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a00ce7-6c8c-4ba0-9d36-a63252b84c1b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:04:57.552601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e4a00ce7-6c8c-4ba0-9d36-a63252b84c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:57.613749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:57.657027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a00ce7-6c8c-4ba0-9d36-a63252b84c1b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:04:57.729911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284720e-d623-427d-8bce-f6e69b45bbd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:04:57.730607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284720e-d623-427d-8bce-f6e69b45bbd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:04:57.731517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284720e-d623-427d-8bce-f6e69b45bbd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:04:57.731780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284720e-d623-427d-8bce-f6e69b45bbd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:04:57.734753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2284720e-d623-427d-8bce-f6e69b45bbd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:04:57.839866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:04:57.842311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284720e-d623-427d-8bce-f6e69b45bbd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:04:57.843787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284720e-d623-427d-8bce-f6e69b45bbd4 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:04:57.861792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 662/1328] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:00.134079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceeaac-32d1-4a96-8d6f-45bcaa76d222 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:00.134609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceeaac-32d1-4a96-8d6f-45bcaa76d222 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:00.134609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceeaac-32d1-4a96-8d6f-45bcaa76d222 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:00.136011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 667/1329] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:00.141979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de44a1-9c25-4314-9f7d-d7148827e570 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:00.142414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de44a1-9c25-4314-9f7d-d7148827e570 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:00.142591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de44a1-9c25-4314-9f7d-d7148827e570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:00.208876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-20de44a1-9c25-4314-9f7d-d7148827e570 None None] MFA Rules not processed for user `90d9e3351e21469a95b1a88347c81225`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:00.210014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-20de44a1-9c25-4314-9f7d-d7148827e570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:00.306497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 663/1330] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:00.344955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceeaac-32d1-4a96-8d6f-45bcaa76d222 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:00.346534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0dceeaac-32d1-4a96-8d6f-45bcaa76d222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:00.376783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:00.450152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceeaac-32d1-4a96-8d6f-45bcaa76d222 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:00.524235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e46c9d-77e9-4677-8fbf-cbafbd03b707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:00.524804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e46c9d-77e9-4677-8fbf-cbafbd03b707 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:00.525086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e46c9d-77e9-4677-8fbf-cbafbd03b707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:00.525386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e46c9d-77e9-4677-8fbf-cbafbd03b707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:00.527020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-08e46c9d-77e9-4677-8fbf-cbafbd03b707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:00.605034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:00.638427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e46c9d-77e9-4677-8fbf-cbafbd03b707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:00.639988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e46c9d-77e9-4677-8fbf-cbafbd03b707 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:00.659945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 668/1331] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:03.339454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22358-2275-42ff-98a8-72ac0f7ceb0b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:03.340601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22358-2275-42ff-98a8-72ac0f7ceb0b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:03.340601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc22358-2275-42ff-98a8-72ac0f7ceb0b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:03.342041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 664/1332] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 10:05:03.347255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221886-d0b3-4863-a88d-a1e267670da8 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:03.347646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221886-d0b3-4863-a88d-a1e267670da8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:03.347811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00221886-d0b3-4863-a88d-a1e267670da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:03.415376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-00221886-d0b3-4863-a88d-a1e267670da8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:03.417417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-00221886-d0b3-4863-a88d-a1e267670da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:03.519030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 669/1333] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:03.555202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc22358-2275-42ff-98a8-72ac0f7ceb0b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:03.556734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-afc22358-2275-42ff-98a8-72ac0f7ceb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:03.628460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:03.662755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc22358-2275-42ff-98a8-72ac0f7ceb0b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:03.744650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626eea0e-8d8b-4b4c-8ece-4f3c4328b96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:03.745335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eea0e-8d8b-4b4c-8ece-4f3c4328b96e service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:03.745887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eea0e-8d8b-4b4c-8ece-4f3c4328b96e service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:03.746239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626eea0e-8d8b-4b4c-8ece-4f3c4328b96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:03.749529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-626eea0e-8d8b-4b4c-8ece-4f3c4328b96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:03.868059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626eea0e-8d8b-4b4c-8ece-4f3c4328b96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:03.869642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626eea0e-8d8b-4b4c-8ece-4f3c4328b96e service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:03.880930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:03.890544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 665/1334] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:06.150538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea375-d3d1-4bf7-9dd1-1fa1272f176a None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:06.151006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea375-d3d1-4bf7-9dd1-1fa1272f176a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:06.151108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea375-d3d1-4bf7-9dd1-1fa1272f176a None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:06.153395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 670/1335] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:06.159492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1155e-8216-49b1-a106-1360be645f1d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:06.159955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1155e-8216-49b1-a106-1360be645f1d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:06.160061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1155e-8216-49b1-a106-1360be645f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:06.224143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9da1155e-8216-49b1-a106-1360be645f1d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:06.225244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9da1155e-8216-49b1-a106-1360be645f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:06.321023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 666/1336] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:06.360876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea375-d3d1-4bf7-9dd1-1fa1272f176a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:06.362480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b93ea375-d3d1-4bf7-9dd1-1fa1272f176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:06.402133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:06.466477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea375-d3d1-4bf7-9dd1-1fa1272f176a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:06.538726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6905225-059c-422e-b548-6d959395044c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:06.539310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6905225-059c-422e-b548-6d959395044c service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:06.539593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6905225-059c-422e-b548-6d959395044c service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:06.539867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6905225-059c-422e-b548-6d959395044c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:06.541541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f6905225-059c-422e-b548-6d959395044c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:06.623499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:06.648348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6905225-059c-422e-b548-6d959395044c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:06.649831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6905225-059c-422e-b548-6d959395044c service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:06.670912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 671/1337] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:09.515102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98497-4418-4b40-a3cd-e0abdacb08a9 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:09.515681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98497-4418-4b40-a3cd-e0abdacb08a9 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:09.515681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b98497-4418-4b40-a3cd-e0abdacb08a9 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:09.517092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 667/1338] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:09.522752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5df52-9722-4eec-9665-678f9ee981c3 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:09.523163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5df52-9722-4eec-9665-678f9ee981c3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:09.523307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb5df52-9722-4eec-9665-678f9ee981c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:09.590767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-dcb5df52-9722-4eec-9665-678f9ee981c3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:09.591966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-dcb5df52-9722-4eec-9665-678f9ee981c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:09.687123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 672/1339] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:09.723157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98497-4418-4b40-a3cd-e0abdacb08a9 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:09.724748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c9b98497-4418-4b40-a3cd-e0abdacb08a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:09.803745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:09.831806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b98497-4418-4b40-a3cd-e0abdacb08a9 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:09.906851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc3306-15b9-4907-b310-ef0aa7ac5f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:09.907551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc3306-15b9-4907-b310-ef0aa7ac5f69 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:09.907865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc3306-15b9-4907-b310-ef0aa7ac5f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:09.908145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc3306-15b9-4907-b310-ef0aa7ac5f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:09.909815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-52bc3306-15b9-4907-b310-ef0aa7ac5f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:10.022411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc3306-15b9-4907-b310-ef0aa7ac5f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:10.023853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc3306-15b9-4907-b310-ef0aa7ac5f69 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:10.042790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 668/1340] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:14.630609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83493-ae0d-43ae-9dbe-a5d8709e58f9 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:14.630609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83493-ae0d-43ae-9dbe-a5d8709e58f9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:14.630609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f83493-ae0d-43ae-9dbe-a5d8709e58f9 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:14.632115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 673/1341] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:14.637784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8776aef-8ad8-49d5-8f15-04bd4cb87bcb None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:14.642291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8776aef-8ad8-49d5-8f15-04bd4cb87bcb None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:14.642291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8776aef-8ad8-49d5-8f15-04bd4cb87bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:14.644439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:14.714441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d8776aef-8ad8-49d5-8f15-04bd4cb87bcb None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:14.716296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d8776aef-8ad8-49d5-8f15-04bd4cb87bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:14.827163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 669/1342] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:05:14.866146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83493-ae0d-43ae-9dbe-a5d8709e58f9 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:14.868264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-52f83493-ae0d-43ae-9dbe-a5d8709e58f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:14.920846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:14.964011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f83493-ae0d-43ae-9dbe-a5d8709e58f9 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:14.968314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc6baf-0b98-4128-880f-6f84c1c1c489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:14.968964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc6baf-0b98-4128-880f-6f84c1c1c489 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:14.969248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc6baf-0b98-4128-880f-6f84c1c1c489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:14.969552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc6baf-0b98-4128-880f-6f84c1c1c489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:14.978740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc6baf-0b98-4128-880f-6f84c1c1c489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:14.980204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc6baf-0b98-4128-880f-6f84c1c1c489 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:14.990267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 674/1343] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:18.081305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05933c6-a10c-4f56-a216-bb9a3d0b3c15 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:18.081842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05933c6-a10c-4f56-a216-bb9a3d0b3c15 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:18.081842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05933c6-a10c-4f56-a216-bb9a3d0b3c15 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:18.083148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 670/1344] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:18.088392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30a2c-2ae1-4840-a078-9ad22e1294f0 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:18.088826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30a2c-2ae1-4840-a078-9ad22e1294f0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:18.089060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c30a2c-2ae1-4840-a078-9ad22e1294f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:18.116821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-29c30a2c-2ae1-4840-a078-9ad22e1294f0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:18.118046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-29c30a2c-2ae1-4840-a078-9ad22e1294f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:18.198102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 675/1345] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:18.235073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05933c6-a10c-4f56-a216-bb9a3d0b3c15 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:18.245842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05933c6-a10c-4f56-a216-bb9a3d0b3c15 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:18.250294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfb2d47-f5da-48c9-9bcf-4d31c76d4253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:18.250927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2d47-f5da-48c9-9bcf-4d31c76d4253 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:18.251205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2d47-f5da-48c9-9bcf-4d31c76d4253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:18.251478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfb2d47-f5da-48c9-9bcf-4d31c76d4253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:18.260412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb2d47-f5da-48c9-9bcf-4d31c76d4253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:18.262014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfb2d47-f5da-48c9-9bcf-4d31c76d4253 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:18.271502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 671/1346] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:22.890780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f614934-99f6-40c4-a251-2b82ac36c783 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:22.891313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f614934-99f6-40c4-a251-2b82ac36c783 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:22.891313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f614934-99f6-40c4-a251-2b82ac36c783 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:22.893003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 676/1347] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 10:05:22.898553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84794e78-94f5-4e52-b0c6-bacde45e8262 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:22.899003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84794e78-94f5-4e52-b0c6-bacde45e8262 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:22.899097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84794e78-94f5-4e52-b0c6-bacde45e8262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:22.918172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-84794e78-94f5-4e52-b0c6-bacde45e8262 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:22.919445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-84794e78-94f5-4e52-b0c6-bacde45e8262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:22.927525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 672/1348] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:22.966470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f614934-99f6-40c4-a251-2b82ac36c783 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:22.977100 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f614934-99f6-40c4-a251-2b82ac36c783 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:22.981293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44836c-2109-41da-9cb6-743427cd7826 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:22.982010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44836c-2109-41da-9cb6-743427cd7826 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:22.982291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44836c-2109-41da-9cb6-743427cd7826 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:22.982566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44836c-2109-41da-9cb6-743427cd7826 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:22.991932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44836c-2109-41da-9cb6-743427cd7826 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:22.993479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44836c-2109-41da-9cb6-743427cd7826 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:23.002934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 677/1349] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:28.369900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301466b8-7c91-4f11-a83f-676a8f10d660 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:28.370926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301466b8-7c91-4f11-a83f-676a8f10d660 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:28.371880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301466b8-7c91-4f11-a83f-676a8f10d660 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:28.373773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 673/1350] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:05:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:28.379050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eee4b4-e555-49c0-a2d4-613b297ca007 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:28.379957 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eee4b4-e555-49c0-a2d4-613b297ca007 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:28.380473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eee4b4-e555-49c0-a2d4-613b297ca007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:28.459503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-04eee4b4-e555-49c0-a2d4-613b297ca007 None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:28.460953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-04eee4b4-e555-49c0-a2d4-613b297ca007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:28.489249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:28.566625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 678/1351] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:05:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:05:32.783114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3668-302f-474f-a9a3-0afa4220dfec None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:32.783626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3668-302f-474f-a9a3-0afa4220dfec None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:32.783626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a3668-302f-474f-a9a3-0afa4220dfec None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:32.784932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 674/1352] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:32.790094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add8f15-8a75-4ebc-9070-f1b38cdcbe7e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:32.790516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add8f15-8a75-4ebc-9070-f1b38cdcbe7e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:32.790648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5add8f15-8a75-4ebc-9070-f1b38cdcbe7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:32.810886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5add8f15-8a75-4ebc-9070-f1b38cdcbe7e None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:32.812076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5add8f15-8a75-4ebc-9070-f1b38cdcbe7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:32.820870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 679/1353] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:05:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:32.912853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=72 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:34.603309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc631b-2040-4cd3-b727-53e6c56faba2 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:34.603957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc631b-2040-4cd3-b727-53e6c56faba2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:34.603957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc631b-2040-4cd3-b727-53e6c56faba2 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:34.605371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 675/1354] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:34.611080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eead2-116e-4d69-9bce-319b57bffcda None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:34.611494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eead2-116e-4d69-9bce-319b57bffcda None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:34.611667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6eead2-116e-4d69-9bce-319b57bffcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:34.630598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4c6eead2-116e-4d69-9bce-319b57bffcda None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:34.631773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4c6eead2-116e-4d69-9bce-319b57bffcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:34.640469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 680/1355] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:05:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:05:38.435973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa50b5-9603-422f-9ff0-d03e4e2663e6 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:38.436490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa50b5-9603-422f-9ff0-d03e4e2663e6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:38.436573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa50b5-9603-422f-9ff0-d03e4e2663e6 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:38.438171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 676/1356] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 10:05:38.443708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6defc5-96df-4478-aac5-15deffcd82e0 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:38.444235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6defc5-96df-4478-aac5-15deffcd82e0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:38.444468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6defc5-96df-4478-aac5-15deffcd82e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:38.463740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ae6defc5-96df-4478-aac5-15deffcd82e0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:38.465398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ae6defc5-96df-4478-aac5-15deffcd82e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:38.473970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 681/1357] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:38.533060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77cd9-7d33-43b6-a386-4f75068dd133 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:38.533729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77cd9-7d33-43b6-a386-4f75068dd133 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:38.534411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e77cd9-7d33-43b6-a386-4f75068dd133 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:38.540387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 677/1358] 10.222.0.35 () {54 vars in 961 bytes} [Tue May 24 10:05:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:38.547265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f932fc-ca9b-40bd-acc9-9b5c1d3cd23c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:38.547634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f932fc-ca9b-40bd-acc9-9b5c1d3cd23c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:38.547919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f932fc-ca9b-40bd-acc9-9b5c1d3cd23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:38.611739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e6f932fc-ca9b-40bd-acc9-9b5c1d3cd23c None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:38.613088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e6f932fc-ca9b-40bd-acc9-9b5c1d3cd23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:38.706378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 682/1359] 10.222.0.35 () {58 vars in 1058 bytes} [Tue May 24 10:05:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:05:38.714249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e77cd9-7d33-43b6-a386-4f75068dd133 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:38.729599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e77cd9-7d33-43b6-a386-4f75068dd133 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:38.735350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0ef341-f328-4277-a5d2-09449f4242f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:38.735986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ef341-f328-4277-a5d2-09449f4242f0 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:38.736266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ef341-f328-4277-a5d2-09449f4242f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:38.736542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0ef341-f328-4277-a5d2-09449f4242f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:38.745040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ef341-f328-4277-a5d2-09449f4242f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:38.747613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0ef341-f328-4277-a5d2-09449f4242f0 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:38.757559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 678/1360] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:41.204884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b76c-427b-4e72-9088-0addc78d09e2 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:41.205464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b76c-427b-4e72-9088-0addc78d09e2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:41.205588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14b76c-427b-4e72-9088-0addc78d09e2 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:41.207055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 683/1361] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:41.212260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a788-fab4-4d7e-968d-ca156f8e0c71 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:41.212729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a788-fab4-4d7e-968d-ca156f8e0c71 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:41.212814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84a788-fab4-4d7e-968d-ca156f8e0c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:41.232156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ac84a788-fab4-4d7e-968d-ca156f8e0c71 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:41.233454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ac84a788-fab4-4d7e-968d-ca156f8e0c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:41.244211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 679/1362] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:41.285581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b76c-427b-4e72-9088-0addc78d09e2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:41.301628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14b76c-427b-4e72-9088-0addc78d09e2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:41.307395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab587f-59fd-4b3b-87d1-9fffa6921ae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:41.308148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab587f-59fd-4b3b-87d1-9fffa6921ae1 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:41.308651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab587f-59fd-4b3b-87d1-9fffa6921ae1 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:41.309185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab587f-59fd-4b3b-87d1-9fffa6921ae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:41.318108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab587f-59fd-4b3b-87d1-9fffa6921ae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:41.319902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab587f-59fd-4b3b-87d1-9fffa6921ae1 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:41.330607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 684/1363] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:44.038694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad34ca-e5b7-448a-8215-435f4d0d6997 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:44.039592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad34ca-e5b7-448a-8215-435f4d0d6997 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:44.039878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad34ca-e5b7-448a-8215-435f4d0d6997 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:44.041339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 680/1364] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:44.047527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc92c-b415-4b34-bc11-62aae726c7e1 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:44.047949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc92c-b415-4b34-bc11-62aae726c7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:44.048195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc92c-b415-4b34-bc11-62aae726c7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:44.066684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-89fdc92c-b415-4b34-bc11-62aae726c7e1 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:44.067959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-89fdc92c-b415-4b34-bc11-62aae726c7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:44.080493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 685/1365] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:44.127623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337628b-a160-4fd9-8378-6ebd7dc35c0e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:44.127945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337628b-a160-4fd9-8378-6ebd7dc35c0e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:44.128248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a337628b-a160-4fd9-8378-6ebd7dc35c0e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:44.129662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 681/1366] 10.222.0.35 () {54 vars in 961 bytes} [Tue May 24 10:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:44.136162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7d0f2-5942-4fbc-b9b2-10c713f01b6f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:44.136558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7d0f2-5942-4fbc-b9b2-10c713f01b6f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:44.136821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7d0f2-5942-4fbc-b9b2-10c713f01b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:44.157625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-79b7d0f2-5942-4fbc-b9b2-10c713f01b6f None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:44.159164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-79b7d0f2-5942-4fbc-b9b2-10c713f01b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:44.170388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 686/1367] 10.222.0.35 () {58 vars in 1058 bytes} [Tue May 24 10:05:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:44.179090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337628b-a160-4fd9-8378-6ebd7dc35c0e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:44.206952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a337628b-a160-4fd9-8378-6ebd7dc35c0e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:44.212860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef4ef0-e3d6-47d8-b267-eda3c6bcbe83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:44.218152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef4ef0-e3d6-47d8-b267-eda3c6bcbe83 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:44.218152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef4ef0-e3d6-47d8-b267-eda3c6bcbe83 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:44.218152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef4ef0-e3d6-47d8-b267-eda3c6bcbe83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:44.225237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef4ef0-e3d6-47d8-b267-eda3c6bcbe83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:44.226905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef4ef0-e3d6-47d8-b267-eda3c6bcbe83 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:44.237574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 682/1368] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:46.678861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce5cd-8edc-4002-aef8-e7a6e74e8fba None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:46.679515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce5cd-8edc-4002-aef8-e7a6e74e8fba None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:46.679515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ce5cd-8edc-4002-aef8-e7a6e74e8fba None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:46.680778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 687/1369] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:46.687089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02636e1b-1b11-4135-87d1-c0224728cc70 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:46.687554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02636e1b-1b11-4135-87d1-c0224728cc70 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:46.687953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02636e1b-1b11-4135-87d1-c0224728cc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:46.708924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-02636e1b-1b11-4135-87d1-c0224728cc70 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:46.710509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-02636e1b-1b11-4135-87d1-c0224728cc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:46.720431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 683/1370] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:46.764264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce5cd-8edc-4002-aef8-e7a6e74e8fba None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:46.779191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ce5cd-8edc-4002-aef8-e7a6e74e8fba None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:46.784823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3642dce-e7fd-4e54-911f-deb7b06fae0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:46.785721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3642dce-e7fd-4e54-911f-deb7b06fae0e service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:46.786003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3642dce-e7fd-4e54-911f-deb7b06fae0e service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:46.786272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3642dce-e7fd-4e54-911f-deb7b06fae0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:46.796223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3642dce-e7fd-4e54-911f-deb7b06fae0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:46.800742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3642dce-e7fd-4e54-911f-deb7b06fae0e service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:46.809338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 688/1371] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:46.898737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9292a-25b1-45d3-926a-a6c7ca9f35c6 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:46.899131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9292a-25b1-45d3-926a-a6c7ca9f35c6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:46.899513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba9292a-25b1-45d3-926a-a6c7ca9f35c6 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:46.900971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 684/1372] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:46.907196 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2491e7-c5de-40ae-9a1e-27a07b7089cc None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:46.908561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2491e7-c5de-40ae-9a1e-27a07b7089cc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:46.908561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2491e7-c5de-40ae-9a1e-27a07b7089cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:46.926751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ee2491e7-c5de-40ae-9a1e-27a07b7089cc None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:46.927953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ee2491e7-c5de-40ae-9a1e-27a07b7089cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:46.937323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 689/1373] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:47.263595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5eae-9f90-4ab8-8b83-84ac2df2946a None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:47.265015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5eae-9f90-4ab8-8b83-84ac2df2946a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:47.265015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e5eae-9f90-4ab8-8b83-84ac2df2946a None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:47.265766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 685/1374] 10.222.0.35 () {54 vars in 963 bytes} [Tue May 24 10:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:47.272349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2050-e3fd-4100-86fd-24dc6d6e9a4f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:47.272791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2050-e3fd-4100-86fd-24dc6d6e9a4f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:47.273174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b2050-e3fd-4100-86fd-24dc6d6e9a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:47.283157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:05:47.332705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-966b2050-e3fd-4100-86fd-24dc6d6e9a4f None None] MFA Rules not processed for user `aaba1a8ba921489e96f9fa4acecffb74`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:47.334103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-966b2050-e3fd-4100-86fd-24dc6d6e9a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:47.399810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 690/1375] 10.222.0.35 () {58 vars in 1060 bytes} [Tue May 24 10:05:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:05:47.409275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5eae-9f90-4ab8-8b83-84ac2df2946a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:47.420942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e5eae-9f90-4ab8-8b83-84ac2df2946a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:47.424624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8548be-77cf-40f5-b45e-d3ede7742955 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:47.425428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8548be-77cf-40f5-b45e-d3ede7742955 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:47.425818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8548be-77cf-40f5-b45e-d3ede7742955 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:47.426199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8548be-77cf-40f5-b45e-d3ede7742955 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:47.436248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8548be-77cf-40f5-b45e-d3ede7742955 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:47.437906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8548be-77cf-40f5-b45e-d3ede7742955 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:47.448441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 686/1376] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:49.333052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea2931-162b-4fd7-be6a-8b7a06111461 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:49.333627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea2931-162b-4fd7-be6a-8b7a06111461 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:49.333627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ea2931-162b-4fd7-be6a-8b7a06111461 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:49.335026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 691/1377] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:49.341534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ea73e-9b5e-4a9f-bfe9-0380809f2cbd None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:49.342098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ea73e-9b5e-4a9f-bfe9-0380809f2cbd None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:49.342197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ea73e-9b5e-4a9f-bfe9-0380809f2cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:49.362117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9a7ea73e-9b5e-4a9f-bfe9-0380809f2cbd None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:49.363342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9a7ea73e-9b5e-4a9f-bfe9-0380809f2cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:49.372070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 687/1378] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:49.415871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea2931-162b-4fd7-be6a-8b7a06111461 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:49.431109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ea2931-162b-4fd7-be6a-8b7a06111461 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:49.437323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19f639-633a-4580-b6e0-7d269af0030c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:49.437998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19f639-633a-4580-b6e0-7d269af0030c service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:49.438278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19f639-633a-4580-b6e0-7d269af0030c service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:49.438550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19f639-633a-4580-b6e0-7d269af0030c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:49.447901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19f639-633a-4580-b6e0-7d269af0030c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:49.450491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b19f639-633a-4580-b6e0-7d269af0030c service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:49.460439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 692/1379] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:51.889095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9d1c-2343-4892-8bb3-2be595c767cf None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:51.889653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9d1c-2343-4892-8bb3-2be595c767cf None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:51.889653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a9d1c-2343-4892-8bb3-2be595c767cf None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:51.890941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 688/1380] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:51.896085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a3f37-651e-4e65-be67-bf3295f64ab1 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:51.896502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a3f37-651e-4e65-be67-bf3295f64ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:51.896638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a3f37-651e-4e65-be67-bf3295f64ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:51.916219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-be8a3f37-651e-4e65-be67-bf3295f64ab1 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:51.917417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-be8a3f37-651e-4e65-be67-bf3295f64ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:51.929309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 693/1381] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:51.969665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a9d1c-2343-4892-8bb3-2be595c767cf None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:51.983941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a9d1c-2343-4892-8bb3-2be595c767cf None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:51.989159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b101ac-dad7-44a6-aff1-a48d9558c426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:51.989817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b101ac-dad7-44a6-aff1-a48d9558c426 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:51.990100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b101ac-dad7-44a6-aff1-a48d9558c426 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:51.990378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b101ac-dad7-44a6-aff1-a48d9558c426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:51.999134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b101ac-dad7-44a6-aff1-a48d9558c426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:52.000713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b101ac-dad7-44a6-aff1-a48d9558c426 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:52.010473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 689/1382] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:54.388606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944d8b4-3d54-4371-ade8-6d64821a4890 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:54.389506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944d8b4-3d54-4371-ade8-6d64821a4890 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:54.390028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944d8b4-3d54-4371-ade8-6d64821a4890 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:54.391633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 694/1383] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:54.397067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77265bdc-030a-4f28-af7c-fe0bfe80721f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:54.397548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77265bdc-030a-4f28-af7c-fe0bfe80721f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:54.397675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77265bdc-030a-4f28-af7c-fe0bfe80721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:54.415686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-77265bdc-030a-4f28-af7c-fe0bfe80721f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:54.416781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-77265bdc-030a-4f28-af7c-fe0bfe80721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:54.425259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 690/1384] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:54.466370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944d8b4-3d54-4371-ade8-6d64821a4890 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:54.483799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944d8b4-3d54-4371-ade8-6d64821a4890 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:54.490120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c1ed8-824d-458f-abc5-68fcd995bdf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:54.490861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c1ed8-824d-458f-abc5-68fcd995bdf4 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:54.491302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c1ed8-824d-458f-abc5-68fcd995bdf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:54.492465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c1ed8-824d-458f-abc5-68fcd995bdf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:54.502469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c1ed8-824d-458f-abc5-68fcd995bdf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:54.504280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c1ed8-824d-458f-abc5-68fcd995bdf4 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:54.515691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 695/1385] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:57.103483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214adb6-b467-4168-bdf0-7ab6805a5f67 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:57.104055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214adb6-b467-4168-bdf0-7ab6805a5f67 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:57.104055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214adb6-b467-4168-bdf0-7ab6805a5f67 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:57.105586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 691/1386] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:57.111553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4511-90c7-41b8-bc5c-1593a9759b5a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:57.112135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4511-90c7-41b8-bc5c-1593a9759b5a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:57.112135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ca4511-90c7-41b8-bc5c-1593a9759b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:57.130919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-67ca4511-90c7-41b8-bc5c-1593a9759b5a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:57.132185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-67ca4511-90c7-41b8-bc5c-1593a9759b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:57.140957 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 696/1387] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:57.187121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214adb6-b467-4168-bdf0-7ab6805a5f67 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:57.202697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214adb6-b467-4168-bdf0-7ab6805a5f67 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:57.208335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a722a34-9286-48b7-a256-21ab986d0e74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:57.209002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722a34-9286-48b7-a256-21ab986d0e74 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:57.209327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722a34-9286-48b7-a256-21ab986d0e74 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:57.209625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a722a34-9286-48b7-a256-21ab986d0e74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:57.219007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a722a34-9286-48b7-a256-21ab986d0e74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:57.220664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a722a34-9286-48b7-a256-21ab986d0e74 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:57.230679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 692/1388] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:05:59.733693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a01277-c0a2-4647-8463-d4e75d8d096c None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:59.734255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a01277-c0a2-4647-8463-d4e75d8d096c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:59.734433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a01277-c0a2-4647-8463-d4e75d8d096c None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:59.736319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 697/1389] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:05:59.741813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0ec3-d4b0-4bbb-ae9d-4464f0d84928 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:59.742261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0ec3-d4b0-4bbb-ae9d-4464f0d84928 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:59.742409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0ec3-d4b0-4bbb-ae9d-4464f0d84928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:59.760407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-7a9c0ec3-d4b0-4bbb-ae9d-4464f0d84928 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:05:59.761719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7a9c0ec3-d4b0-4bbb-ae9d-4464f0d84928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:05:59.769933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 693/1390] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:05:59.820587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a01277-c0a2-4647-8463-d4e75d8d096c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:05:59.839321 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a01277-c0a2-4647-8463-d4e75d8d096c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:05:59.847377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650ebb0c-00fa-440c-a440-2016a8fda138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:05:59.848208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ebb0c-00fa-440c-a440-2016a8fda138 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:05:59.848575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ebb0c-00fa-440c-a440-2016a8fda138 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:05:59.849122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650ebb0c-00fa-440c-a440-2016a8fda138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:05:59.860910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ebb0c-00fa-440c-a440-2016a8fda138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:05:59.863001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650ebb0c-00fa-440c-a440-2016a8fda138 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:05:59.875558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 698/1391] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:05:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:06:02.535208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b90fa9-ab00-41cc-ac46-ddda0f336e6a None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:02.535822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b90fa9-ab00-41cc-ac46-ddda0f336e6a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:02.535822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b90fa9-ab00-41cc-ac46-ddda0f336e6a None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:02.537134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 694/1392] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:02.542349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8e670-5735-45bf-b0d4-2b5372ec6d8f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:02.543373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8e670-5735-45bf-b0d4-2b5372ec6d8f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:02.543496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8e670-5735-45bf-b0d4-2b5372ec6d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:02.562283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5cd8e670-5735-45bf-b0d4-2b5372ec6d8f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:02.563466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5cd8e670-5735-45bf-b0d4-2b5372ec6d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:02.571191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 699/1393] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:02.615042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b90fa9-ab00-41cc-ac46-ddda0f336e6a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:02.630389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b90fa9-ab00-41cc-ac46-ddda0f336e6a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:02.635870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a8251-9c65-4f6d-b8ac-813dcaacdee1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:02.636535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a8251-9c65-4f6d-b8ac-813dcaacdee1 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:02.636931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a8251-9c65-4f6d-b8ac-813dcaacdee1 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:02.637212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a8251-9c65-4f6d-b8ac-813dcaacdee1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:02.646013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a8251-9c65-4f6d-b8ac-813dcaacdee1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:02.647642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a8251-9c65-4f6d-b8ac-813dcaacdee1 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:02.657322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 695/1394] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:03.343873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1c0a0-4af6-4be0-a2b2-ee7dc274692d None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:03.344156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1c0a0-4af6-4be0-a2b2-ee7dc274692d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:03.344430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1c0a0-4af6-4be0-a2b2-ee7dc274692d None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:03.345885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 700/1395] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:03.351279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a689882-82d6-4d0a-8c78-f664c7ab0d0f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:03.351602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a689882-82d6-4d0a-8c78-f664c7ab0d0f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:03.351908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a689882-82d6-4d0a-8c78-f664c7ab0d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:03.358601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=52 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:06:03.370566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9a689882-82d6-4d0a-8c78-f664c7ab0d0f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:03.375465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9a689882-82d6-4d0a-8c78-f664c7ab0d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:03.380491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 696/1396] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:06:03.438588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbc917-37bf-4fa1-a1f0-b27a219fc9c0 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:03.438877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbc917-37bf-4fa1-a1f0-b27a219fc9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:03.439154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bbc917-37bf-4fa1-a1f0-b27a219fc9c0 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:03.440529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 701/1397] 10.222.0.35 () {54 vars in 962 bytes} [Tue May 24 10:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:03.445769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddb7c9-0f80-41be-babf-7912be34fba3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:03.446136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddb7c9-0f80-41be-babf-7912be34fba3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:03.446481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ddb7c9-0f80-41be-babf-7912be34fba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:03.495180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-22ddb7c9-0f80-41be-babf-7912be34fba3 None None] MFA Rules not processed for user `9af9eae0fa5b437e96c1c732dfe4fa7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:03.496443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-22ddb7c9-0f80-41be-babf-7912be34fba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:03.555206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 697/1398] 10.222.0.35 () {58 vars in 1059 bytes} [Tue May 24 10:06:03 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:03.563217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbc917-37bf-4fa1-a1f0-b27a219fc9c0 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:03.573858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bbc917-37bf-4fa1-a1f0-b27a219fc9c0 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:03.577475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08029845-e536-4afe-ab2f-3d00489d0c08 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:03.578111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08029845-e536-4afe-ab2f-3d00489d0c08 service glance] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:03.578398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08029845-e536-4afe-ab2f-3d00489d0c08 service glance] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:03.578668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08029845-e536-4afe-ab2f-3d00489d0c08 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:03.587518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08029845-e536-4afe-ab2f-3d00489d0c08 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:03.588952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08029845-e536-4afe-ab2f-3d00489d0c08 service glance] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:03.599146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 702/1399] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:04.259782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003c422-e37d-4000-b45e-98d71b826885 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:04.260006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003c422-e37d-4000-b45e-98d71b826885 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:04.260284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b003c422-e37d-4000-b45e-98d71b826885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:04.310023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b003c422-e37d-4000-b45e-98d71b826885 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:04.311206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b003c422-e37d-4000-b45e-98d71b826885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:04.374846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 698/1400] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:06:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:04.416477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76e2e-0675-44eb-8f71-240722826890 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:04.416854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76e2e-0675-44eb-8f71-240722826890 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:04.417199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be76e2e-0675-44eb-8f71-240722826890 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:04.419020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 703/1401] 10.222.0.35 () {54 vars in 942 bytes} [Tue May 24 10:06:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:04.425542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407da17e-061d-4416-b76e-0f55d67bf6c5 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:04.425845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407da17e-061d-4416-b76e-0f55d67bf6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:04.426155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407da17e-061d-4416-b76e-0f55d67bf6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:04.474776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-407da17e-061d-4416-b76e-0f55d67bf6c5 None None] MFA Rules not processed for user `05f46d019e1348adabd023de481f0a3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:04.475961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-407da17e-061d-4416-b76e-0f55d67bf6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:04.488505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:06:04.549403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 699/1402] 10.222.0.35 () {58 vars in 1039 bytes} [Tue May 24 10:06:04 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:04.559568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be76e2e-0675-44eb-8f71-240722826890 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:04.572774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be76e2e-0675-44eb-8f71-240722826890 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:04.579133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6666e4-b4ca-4fd4-892e-176cd10a0e3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:04.579942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6666e4-b4ca-4fd4-892e-176cd10a0e3e service swift] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:04.580236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6666e4-b4ca-4fd4-892e-176cd10a0e3e service swift] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:04.580532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6666e4-b4ca-4fd4-892e-176cd10a0e3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:04.591647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6666e4-b4ca-4fd4-892e-176cd10a0e3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:04.593242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6666e4-b4ca-4fd4-892e-176cd10a0e3e service swift] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:04.603200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 704/1403] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:06:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:06:05.114044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97ef8-fb66-4958-a73a-8f668510f07c None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:05.114600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97ef8-fb66-4958-a73a-8f668510f07c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:05.114600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97ef8-fb66-4958-a73a-8f668510f07c None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:05.115965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 700/1404] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:05.121888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38f9dc-9949-4e75-a251-be74537df190 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:05.122381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38f9dc-9949-4e75-a251-be74537df190 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:05.122854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38f9dc-9949-4e75-a251-be74537df190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:05.141233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-cf38f9dc-9949-4e75-a251-be74537df190 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:05.142780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cf38f9dc-9949-4e75-a251-be74537df190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:05.151320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 705/1405] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:05.199064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c97ef8-fb66-4958-a73a-8f668510f07c None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:05.214578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c97ef8-fb66-4958-a73a-8f668510f07c None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:05.220817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4e30f-aa04-4669-b69e-ea2cead71903 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:05.221486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e30f-aa04-4669-b69e-ea2cead71903 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:05.221774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e30f-aa04-4669-b69e-ea2cead71903 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:05.222202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e30f-aa04-4669-b69e-ea2cead71903 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:05.233733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4e30f-aa04-4669-b69e-ea2cead71903 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:05.235673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4e30f-aa04-4669-b69e-ea2cead71903 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:05.246385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 701/1406] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:07.764417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d20e4b-2c04-4835-bfb3-5be9e798ae41 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:07.764948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d20e4b-2c04-4835-bfb3-5be9e798ae41 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:07.764948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d20e4b-2c04-4835-bfb3-5be9e798ae41 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:07.766303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 706/1407] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:07.771700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afea661-b456-459c-a367-978578c035c1 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:07.772135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afea661-b456-459c-a367-978578c035c1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:07.772276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afea661-b456-459c-a367-978578c035c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:07.791021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2afea661-b456-459c-a367-978578c035c1 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:07.792237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2afea661-b456-459c-a367-978578c035c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:07.804713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 702/1408] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:07.849183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d20e4b-2c04-4835-bfb3-5be9e798ae41 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:07.866259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d20e4b-2c04-4835-bfb3-5be9e798ae41 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:07.873128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4730ab87-1b3b-4b1b-b0a6-2b45f33b8678 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:07.873879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730ab87-1b3b-4b1b-b0a6-2b45f33b8678 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:07.874251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730ab87-1b3b-4b1b-b0a6-2b45f33b8678 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:07.874937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4730ab87-1b3b-4b1b-b0a6-2b45f33b8678 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:07.884116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730ab87-1b3b-4b1b-b0a6-2b45f33b8678 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:07.885845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4730ab87-1b3b-4b1b-b0a6-2b45f33b8678 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:07.896983 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 707/1409] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:17.785923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=33 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:06:17.892216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=84 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:06:29.462495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0047e-c660-4cd0-b2fe-483d8fe5e250 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:29.463068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0047e-c660-4cd0-b2fe-483d8fe5e250 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:29.463068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0047e-c660-4cd0-b2fe-483d8fe5e250 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:29.464413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 703/1410] 10.222.0.35 () {54 vars in 918 bytes} [Tue May 24 10:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:29.469715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4186c-c09c-40e6-ad95-f920c7604aae None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:29.470183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4186c-c09c-40e6-ad95-f920c7604aae None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:29.470293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4186c-c09c-40e6-ad95-f920c7604aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:29.488507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9bb4186c-c09c-40e6-ad95-f920c7604aae None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:29.489734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9bb4186c-c09c-40e6-ad95-f920c7604aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:29.498947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 708/1411] 10.222.0.35 () {58 vars in 1015 bytes} [Tue May 24 10:06:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:29.513128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dadfb-7cec-4df1-ac07-0f819fd2e06c None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:29.513463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dadfb-7cec-4df1-ac07-0f819fd2e06c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:29.513746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3dadfb-7cec-4df1-ac07-0f819fd2e06c None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:29.515107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 704/1412] 10.222.0.35 () {54 vars in 918 bytes} [Tue May 24 10:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:29.519725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377703e-ab97-4aff-8105-567ebf2abef2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:29.520039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377703e-ab97-4aff-8105-567ebf2abef2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:29.520317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377703e-ab97-4aff-8105-567ebf2abef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:29.538303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b377703e-ab97-4aff-8105-567ebf2abef2 None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:29.539514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b377703e-ab97-4aff-8105-567ebf2abef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:29.549127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 709/1413] 10.222.0.35 () {58 vars in 1015 bytes} [Tue May 24 10:06:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:32.220792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc464-2e85-447b-af88-db011a514dd0 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:32.221387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc464-2e85-447b-af88-db011a514dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:32.221387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fc464-2e85-447b-af88-db011a514dd0 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:32.222717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:32.227994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8f7c-e523-4897-94af-1ac505250711 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:32.228446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8f7c-e523-4897-94af-1ac505250711 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:32.228555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8f7c-e523-4897-94af-1ac505250711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:32.246648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-cfec8f7c-e523-4897-94af-1ac505250711 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:32.247831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cfec8f7c-e523-4897-94af-1ac505250711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:32.255496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:32.299487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc464-2e85-447b-af88-db011a514dd0 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:32.313939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fc464-2e85-447b-af88-db011a514dd0 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:32.319186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6fe2c-dc04-48d7-865c-eae1106866d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:32.319821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6fe2c-dc04-48d7-865c-eae1106866d3 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:32.320104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6fe2c-dc04-48d7-865c-eae1106866d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:32.320409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6fe2c-dc04-48d7-865c-eae1106866d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:32.329175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6fe2c-dc04-48d7-865c-eae1106866d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:32.330784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6fe2c-dc04-48d7-865c-eae1106866d3 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:32.340687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 706/1416] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:33.305585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce727-b974-484f-97f9-fb66a7ae41ad None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:33.306164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce727-b974-484f-97f9-fb66a7ae41ad None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:33.306164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce727-b974-484f-97f9-fb66a7ae41ad None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:33.307520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:33.312585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c780531-cccc-4081-9bff-b9ffcee5e42d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:33.313031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c780531-cccc-4081-9bff-b9ffcee5e42d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:33.313142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c780531-cccc-4081-9bff-b9ffcee5e42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:33.331357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5c780531-cccc-4081-9bff-b9ffcee5e42d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:33.332569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5c780531-cccc-4081-9bff-b9ffcee5e42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:33.340917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:06:33.506523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ce727-b974-484f-97f9-fb66a7ae41ad None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:33.521136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ce727-b974-484f-97f9-fb66a7ae41ad None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:33.529864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302bbfbd-b2da-41b0-be5b-ebd81bbff51c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:33.529864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bbfbd-b2da-41b0-be5b-ebd81bbff51c service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:33.529864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bbfbd-b2da-41b0-be5b-ebd81bbff51c service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:33.529864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302bbfbd-b2da-41b0-be5b-ebd81bbff51c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:33.536803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bbfbd-b2da-41b0-be5b-ebd81bbff51c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:33.538421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302bbfbd-b2da-41b0-be5b-ebd81bbff51c service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:33.547906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 712/1419] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:34.473501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413910-4cdc-48ee-92ec-da07522c7115 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:34.474071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413910-4cdc-48ee-92ec-da07522c7115 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:34.474071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77413910-4cdc-48ee-92ec-da07522c7115 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:34.475462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:34.481095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4fdbe-190b-420f-8397-10a39e0d617b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:34.481689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4fdbe-190b-420f-8397-10a39e0d617b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:34.481689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d4fdbe-190b-420f-8397-10a39e0d617b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:34.499520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e5d4fdbe-190b-420f-8397-10a39e0d617b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:34.500722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e5d4fdbe-190b-420f-8397-10a39e0d617b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:34.508897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:34.660541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77413910-4cdc-48ee-92ec-da07522c7115 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:34.675557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77413910-4cdc-48ee-92ec-da07522c7115 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:34.680791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466549f-8053-4d0e-908b-3a88a423564a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:34.681457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466549f-8053-4d0e-908b-3a88a423564a service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:34.681740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466549f-8053-4d0e-908b-3a88a423564a service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:34.682013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466549f-8053-4d0e-908b-3a88a423564a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:34.690800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466549f-8053-4d0e-908b-3a88a423564a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:34.692369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466549f-8053-4d0e-908b-3a88a423564a service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:34.702308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 709/1422] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:35.603059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a23704-f86e-4eea-bbbe-7b211f6b501e None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:35.603607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a23704-f86e-4eea-bbbe-7b211f6b501e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:35.603607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a23704-f86e-4eea-bbbe-7b211f6b501e None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:35.605003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:35.610446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d1c9-db94-41d6-9467-f553a7795c96 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:35.610857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d1c9-db94-41d6-9467-f553a7795c96 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:35.610999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6d1c9-db94-41d6-9467-f553a7795c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:35.628574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-56b6d1c9-db94-41d6-9467-f553a7795c96 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:35.633730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-56b6d1c9-db94-41d6-9467-f553a7795c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:35.637975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:35.788385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a23704-f86e-4eea-bbbe-7b211f6b501e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:35.802996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a23704-f86e-4eea-bbbe-7b211f6b501e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:35.808310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f2fff-e046-4efb-ac70-f6b2fdf54840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:35.809514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f2fff-e046-4efb-ac70-f6b2fdf54840 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:35.809795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f2fff-e046-4efb-ac70-f6b2fdf54840 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:35.810064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f2fff-e046-4efb-ac70-f6b2fdf54840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:35.818228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f2fff-e046-4efb-ac70-f6b2fdf54840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:35.819829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f2fff-e046-4efb-ac70-f6b2fdf54840 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:35.829748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 715/1425] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:36.726163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103df12-6161-4684-9148-19d04d8c41dd None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:36.726731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103df12-6161-4684-9148-19d04d8c41dd None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:36.726731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103df12-6161-4684-9148-19d04d8c41dd None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:36.728096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:36.733638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934810c5-353a-434e-b174-d2a988e56c8e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:36.734083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934810c5-353a-434e-b174-d2a988e56c8e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:36.734195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934810c5-353a-434e-b174-d2a988e56c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:36.752357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-934810c5-353a-434e-b174-d2a988e56c8e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:36.753587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-934810c5-353a-434e-b174-d2a988e56c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:36.763154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:36.911531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103df12-6161-4684-9148-19d04d8c41dd None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:36.926726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103df12-6161-4684-9148-19d04d8c41dd None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:36.932014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270bc7b-52d6-4d1c-8104-32d1c78c44f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:36.932659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270bc7b-52d6-4d1c-8104-32d1c78c44f9 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:36.932940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270bc7b-52d6-4d1c-8104-32d1c78c44f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:36.933213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270bc7b-52d6-4d1c-8104-32d1c78c44f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:36.942149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270bc7b-52d6-4d1c-8104-32d1c78c44f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:36.943733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270bc7b-52d6-4d1c-8104-32d1c78c44f9 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:36.954252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 712/1428] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:37.885722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c17462-2535-4cb5-ba75-56be425b8d35 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:37.886274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c17462-2535-4cb5-ba75-56be425b8d35 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:37.886274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c17462-2535-4cb5-ba75-56be425b8d35 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:37.888409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:37.893316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b9125-b0c0-4e8a-9cc8-2381d746b570 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:37.893748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b9125-b0c0-4e8a-9cc8-2381d746b570 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:37.893883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b9125-b0c0-4e8a-9cc8-2381d746b570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:37.912493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-474b9125-b0c0-4e8a-9cc8-2381d746b570 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:37.913712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-474b9125-b0c0-4e8a-9cc8-2381d746b570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:37.921632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:38.070172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c17462-2535-4cb5-ba75-56be425b8d35 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:38.084723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c17462-2535-4cb5-ba75-56be425b8d35 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:38.090546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5b5f5-2554-441f-9472-22ad1f5fe4bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:38.091176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b5f5-2554-441f-9472-22ad1f5fe4bb service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:38.091456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b5f5-2554-441f-9472-22ad1f5fe4bb service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:38.091729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b5f5-2554-441f-9472-22ad1f5fe4bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:38.099588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5b5f5-2554-441f-9472-22ad1f5fe4bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:38.101159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5b5f5-2554-441f-9472-22ad1f5fe4bb service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:38.110927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 718/1431] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:39.028836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f2833-1718-4dbd-bdd9-058519986d53 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:39.029400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f2833-1718-4dbd-bdd9-058519986d53 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:39.029400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f2833-1718-4dbd-bdd9-058519986d53 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:39.030704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:39.035950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e2b8-332a-486b-aeaa-e8129d600553 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:39.036417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e2b8-332a-486b-aeaa-e8129d600553 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:39.036536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe19e2b8-332a-486b-aeaa-e8129d600553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:39.054048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fe19e2b8-332a-486b-aeaa-e8129d600553 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:39.055218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fe19e2b8-332a-486b-aeaa-e8129d600553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:39.063870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:39.213719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f2833-1718-4dbd-bdd9-058519986d53 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:39.228058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f2833-1718-4dbd-bdd9-058519986d53 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:39.234357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63e2c6-4852-4b84-a82d-54dffcc670e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:39.234997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63e2c6-4852-4b84-a82d-54dffcc670e3 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:39.235280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63e2c6-4852-4b84-a82d-54dffcc670e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:39.235558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63e2c6-4852-4b84-a82d-54dffcc670e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:39.243731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63e2c6-4852-4b84-a82d-54dffcc670e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:39.245345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63e2c6-4852-4b84-a82d-54dffcc670e3 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:39.255623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 715/1434] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:40.172236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42239686-c8c8-4bc1-8256-01047077e9f6 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:40.172689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42239686-c8c8-4bc1-8256-01047077e9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:40.172795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42239686-c8c8-4bc1-8256-01047077e9f6 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:40.174202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:40.179338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e19c-c10b-4586-b22e-b9730da0585d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:40.179736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e19c-c10b-4586-b22e-b9730da0585d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:40.179896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1e19c-c10b-4586-b22e-b9730da0585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:40.198113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4ec1e19c-c10b-4586-b22e-b9730da0585d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:40.199452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4ec1e19c-c10b-4586-b22e-b9730da0585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:40.207301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:06:40.358711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42239686-c8c8-4bc1-8256-01047077e9f6 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:40.373475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42239686-c8c8-4bc1-8256-01047077e9f6 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:40.378753 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c54558-35fb-4838-a78d-ce7a46675799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:40.379482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c54558-35fb-4838-a78d-ce7a46675799 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:40.379765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c54558-35fb-4838-a78d-ce7a46675799 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:40.380035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c54558-35fb-4838-a78d-ce7a46675799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:40.388866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c54558-35fb-4838-a78d-ce7a46675799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:40.390484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c54558-35fb-4838-a78d-ce7a46675799 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:40.399759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 721/1437] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:41.311859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a1330-986b-4aa9-915e-36dbeb8fcd79 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:41.312430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a1330-986b-4aa9-915e-36dbeb8fcd79 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:41.312430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a1330-986b-4aa9-915e-36dbeb8fcd79 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:41.313812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:41.318776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551afd06-a30d-4457-98be-4ab9e4512a5b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:41.319217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551afd06-a30d-4457-98be-4ab9e4512a5b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:41.319333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551afd06-a30d-4457-98be-4ab9e4512a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:41.337969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-551afd06-a30d-4457-98be-4ab9e4512a5b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:41.339156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-551afd06-a30d-4457-98be-4ab9e4512a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:41.346931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:41.497485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a1330-986b-4aa9-915e-36dbeb8fcd79 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:41.512417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a1330-986b-4aa9-915e-36dbeb8fcd79 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:41.517832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518ad1f-98c1-4e48-92c4-cd0dd797790f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:41.518549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518ad1f-98c1-4e48-92c4-cd0dd797790f service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:41.518835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518ad1f-98c1-4e48-92c4-cd0dd797790f service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:41.519115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518ad1f-98c1-4e48-92c4-cd0dd797790f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:41.528418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518ad1f-98c1-4e48-92c4-cd0dd797790f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:41.530040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518ad1f-98c1-4e48-92c4-cd0dd797790f service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:41.540205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 718/1440] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:43.622763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67ef5-52ee-44a7-98a1-ddba8df7cd04 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:43.623320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67ef5-52ee-44a7-98a1-ddba8df7cd04 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:43.623320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b67ef5-52ee-44a7-98a1-ddba8df7cd04 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:43.624684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 10:06:43.630154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c691363-5dd2-43dc-9176-072b9dcb8a9a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:43.631287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c691363-5dd2-43dc-9176-072b9dcb8a9a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:43.631287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c691363-5dd2-43dc-9176-072b9dcb8a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:43.648704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-6c691363-5dd2-43dc-9176-072b9dcb8a9a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:43.649621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6c691363-5dd2-43dc-9176-072b9dcb8a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:43.657646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:43.696868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67ef5-52ee-44a7-98a1-ddba8df7cd04 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:43.707907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b67ef5-52ee-44a7-98a1-ddba8df7cd04 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:43.711443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb535d0-398d-4f04-bb96-bf8dffd61a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:43.712018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb535d0-398d-4f04-bb96-bf8dffd61a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:43.712297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb535d0-398d-4f04-bb96-bf8dffd61a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:43.712571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb535d0-398d-4f04-bb96-bf8dffd61a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:43.721920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb535d0-398d-4f04-bb96-bf8dffd61a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:43.729091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb535d0-398d-4f04-bb96-bf8dffd61a35 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:43.733339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 724/1443] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:46.080942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd9a7f-ad78-45e4-bc22-c4819455d8da None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:46.081484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd9a7f-ad78-45e4-bc22-c4819455d8da None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:46.081581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd9a7f-ad78-45e4-bc22-c4819455d8da None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:46.082889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:46.088080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b1af-db9c-4dcc-b2a8-58648af0acb5 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:46.088534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b1af-db9c-4dcc-b2a8-58648af0acb5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:46.088642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120b1af-db9c-4dcc-b2a8-58648af0acb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:46.106995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b120b1af-db9c-4dcc-b2a8-58648af0acb5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:46.108175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b120b1af-db9c-4dcc-b2a8-58648af0acb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:46.115770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:46.152861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd9a7f-ad78-45e4-bc22-c4819455d8da None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:46.163757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd9a7f-ad78-45e4-bc22-c4819455d8da None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:46.167259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2f9e3-01a7-4dfe-a947-62e04de03f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:46.168383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2f9e3-01a7-4dfe-a947-62e04de03f01 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:46.168668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2f9e3-01a7-4dfe-a947-62e04de03f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:46.168940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2f9e3-01a7-4dfe-a947-62e04de03f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:46.178284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2f9e3-01a7-4dfe-a947-62e04de03f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:46.179737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2f9e3-01a7-4dfe-a947-62e04de03f01 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:46.189927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 721/1446] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:48.747431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e09b8-2fd5-42c4-b113-1672cf050b00 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:48.748099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e09b8-2fd5-42c4-b113-1672cf050b00 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:48.748099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e09b8-2fd5-42c4-b113-1672cf050b00 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:48.749399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:48.754628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b3dad-f181-4113-b043-a99e1676ab6a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:48.755057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b3dad-f181-4113-b043-a99e1676ab6a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:48.755186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b3dad-f181-4113-b043-a99e1676ab6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:48.772711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2c4b3dad-f181-4113-b043-a99e1676ab6a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:48.773926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2c4b3dad-f181-4113-b043-a99e1676ab6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:48.782279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:48.822453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e09b8-2fd5-42c4-b113-1672cf050b00 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:48.833571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e09b8-2fd5-42c4-b113-1672cf050b00 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:48.837033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ea8f5-74c5-4ae0-b22d-55f551e0539a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:48.837704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ea8f5-74c5-4ae0-b22d-55f551e0539a service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:48.837990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ea8f5-74c5-4ae0-b22d-55f551e0539a service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:48.838269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ea8f5-74c5-4ae0-b22d-55f551e0539a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:48.847942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ea8f5-74c5-4ae0-b22d-55f551e0539a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:48.849413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ea8f5-74c5-4ae0-b22d-55f551e0539a service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:48.858829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 727/1449] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:51.165034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3992-3564-4c05-99dc-bb9572b8724f None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:51.165034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3992-3564-4c05-99dc-bb9572b8724f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:51.165034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae3992-3564-4c05-99dc-bb9572b8724f None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:51.166409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:51.172258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1daf55-9373-4d92-ae0c-9ef47c2026fa None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:51.172752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1daf55-9373-4d92-ae0c-9ef47c2026fa None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:51.172890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1daf55-9373-4d92-ae0c-9ef47c2026fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:51.191055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2a1daf55-9373-4d92-ae0c-9ef47c2026fa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:51.192244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2a1daf55-9373-4d92-ae0c-9ef47c2026fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:51.199885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:51.238370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae3992-3564-4c05-99dc-bb9572b8724f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:51.240086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1cae3992-3564-4c05-99dc-bb9572b8724f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:51.253531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae3992-3564-4c05-99dc-bb9572b8724f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:51.256934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bf66f-233c-42bf-8ab4-9e7a1d42fc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:51.257613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf66f-233c-42bf-8ab4-9e7a1d42fc42 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:51.257898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf66f-233c-42bf-8ab4-9e7a1d42fc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:51.258175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf66f-233c-42bf-8ab4-9e7a1d42fc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:51.266893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bf66f-233c-42bf-8ab4-9e7a1d42fc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:51.268322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bf66f-233c-42bf-8ab4-9e7a1d42fc42 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:51.277598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 724/1452] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:06:53.608377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd799d-4ab2-41f3-a9c5-87e33e5ae383 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:53.608925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd799d-4ab2-41f3-a9c5-87e33e5ae383 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:53.608925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd799d-4ab2-41f3-a9c5-87e33e5ae383 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:53.610331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:53.615545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c05ad-ad08-4fe3-8813-bae605fcfdd7 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:53.615989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c05ad-ad08-4fe3-8813-bae605fcfdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:53.616135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6c05ad-ad08-4fe3-8813-bae605fcfdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:53.633594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-6a6c05ad-ad08-4fe3-8813-bae605fcfdd7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:53.634805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6a6c05ad-ad08-4fe3-8813-bae605fcfdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:53.643213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:06:53.681137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd799d-4ab2-41f3-a9c5-87e33e5ae383 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:53.692362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd799d-4ab2-41f3-a9c5-87e33e5ae383 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:53.695798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a40e1-1577-4631-967b-708f99ecf454 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:53.696427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a40e1-1577-4631-967b-708f99ecf454 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:53.696705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a40e1-1577-4631-967b-708f99ecf454 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:53.696981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a40e1-1577-4631-967b-708f99ecf454 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:53.705886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a40e1-1577-4631-967b-708f99ecf454 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:53.707310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a40e1-1577-4631-967b-708f99ecf454 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:53.717743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 730/1455] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:06:55.978646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1fd7-1fe2-47fc-92e7-a5238e364166 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:55.979223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1fd7-1fe2-47fc-92e7-a5238e364166 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:55.979223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1fd7-1fe2-47fc-92e7-a5238e364166 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:55.980996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:55.986278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a701f3-72a2-4521-8f09-6019c2368afa None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:55.986741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a701f3-72a2-4521-8f09-6019c2368afa None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:55.986867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a701f3-72a2-4521-8f09-6019c2368afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:56.004894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-33a701f3-72a2-4521-8f09-6019c2368afa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:56.006110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-33a701f3-72a2-4521-8f09-6019c2368afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:56.014708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:56.051735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d1fd7-1fe2-47fc-92e7-a5238e364166 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:56.063046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d1fd7-1fe2-47fc-92e7-a5238e364166 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:56.066529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcb0ad2-46dd-41c5-9e2a-efdc5a1fb487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:56.067160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb0ad2-46dd-41c5-9e2a-efdc5a1fb487 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:56.067445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb0ad2-46dd-41c5-9e2a-efdc5a1fb487 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:56.067724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcb0ad2-46dd-41c5-9e2a-efdc5a1fb487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:56.077612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb0ad2-46dd-41c5-9e2a-efdc5a1fb487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:56.079070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcb0ad2-46dd-41c5-9e2a-efdc5a1fb487 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:56.088332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 727/1458] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:06:58.340167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0183964-c148-40bc-9467-35dd17fe661e None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:58.340703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0183964-c148-40bc-9467-35dd17fe661e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:58.340703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0183964-c148-40bc-9467-35dd17fe661e None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:58.342131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:06:58.347513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034178f-9409-4b09-b4bc-95dc6c9cd2d7 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:58.347926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034178f-9409-4b09-b4bc-95dc6c9cd2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:58.348098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034178f-9409-4b09-b4bc-95dc6c9cd2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:58.354099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=70 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:06:58.366055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5034178f-9409-4b09-b4bc-95dc6c9cd2d7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:06:58.367257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5034178f-9409-4b09-b4bc-95dc6c9cd2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:06:58.375549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:06:58.413275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0183964-c148-40bc-9467-35dd17fe661e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:06:58.423832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0183964-c148-40bc-9467-35dd17fe661e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:06:58.427265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccae2a-984a-43ce-8b6b-3945814eb0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:06:58.427900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccae2a-984a-43ce-8b6b-3945814eb0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:06:58.428175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccae2a-984a-43ce-8b6b-3945814eb0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:06:58.428447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccae2a-984a-43ce-8b6b-3945814eb0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:06:58.437731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccae2a-984a-43ce-8b6b-3945814eb0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:06:58.439148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccae2a-984a-43ce-8b6b-3945814eb0b3 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:06:58.448244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 733/1461] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:00.693928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7d6ad-fa96-420d-a505-d9be12a065de None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:00.694475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7d6ad-fa96-420d-a505-d9be12a065de None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:00.694475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c7d6ad-fa96-420d-a505-d9be12a065de None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:00.695851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:00.701707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49930bc-5592-46e9-b944-552521047c6a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:00.702163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49930bc-5592-46e9-b944-552521047c6a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:00.702297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49930bc-5592-46e9-b944-552521047c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:00.708383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=79 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:07:00.720648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c49930bc-5592-46e9-b944-552521047c6a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:00.727084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c49930bc-5592-46e9-b944-552521047c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:00.729850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:00.768302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7d6ad-fa96-420d-a505-d9be12a065de None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:00.778813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c7d6ad-fa96-420d-a505-d9be12a065de None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:00.782354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49992562-b71b-4b6e-968c-0916a0f0a96b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:00.782985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49992562-b71b-4b6e-968c-0916a0f0a96b service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:00.783272 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49992562-b71b-4b6e-968c-0916a0f0a96b service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:00.783545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49992562-b71b-4b6e-968c-0916a0f0a96b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:00.792527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49992562-b71b-4b6e-968c-0916a0f0a96b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:00.794094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49992562-b71b-4b6e-968c-0916a0f0a96b service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:00.803413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 730/1464] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:03.064579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5970-ce1d-4b0a-998e-b4d9150168ad None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:03.065466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5970-ce1d-4b0a-998e-b4d9150168ad None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:03.065657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee5970-ce1d-4b0a-998e-b4d9150168ad None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:03.067102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:03.072368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbec3dc-e8c1-48e0-97a7-6ae4cb0bd61e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:03.072826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbec3dc-e8c1-48e0-97a7-6ae4cb0bd61e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:03.072929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbec3dc-e8c1-48e0-97a7-6ae4cb0bd61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:03.091548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1fbec3dc-e8c1-48e0-97a7-6ae4cb0bd61e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:03.092751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1fbec3dc-e8c1-48e0-97a7-6ae4cb0bd61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:03.100538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:03.137969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee5970-ce1d-4b0a-998e-b4d9150168ad None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:03.148828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee5970-ce1d-4b0a-998e-b4d9150168ad None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:03.152373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e17d2-3c5f-443c-b541-c171a9b5f885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:03.153015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e17d2-3c5f-443c-b541-c171a9b5f885 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:03.153412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e17d2-3c5f-443c-b541-c171a9b5f885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:03.153684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e17d2-3c5f-443c-b541-c171a9b5f885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:03.162919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e17d2-3c5f-443c-b541-c171a9b5f885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:03.164354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e17d2-3c5f-443c-b541-c171a9b5f885 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:03.174647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 736/1467] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:07:05.397331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5884721-4e60-4e86-af94-a0ab2219bc0b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:05.397912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5884721-4e60-4e86-af94-a0ab2219bc0b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:05.397912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5884721-4e60-4e86-af94-a0ab2219bc0b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:05.399283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:05.405338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb76e71-aa8b-43e9-8ca2-26684a458a77 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:05.405782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb76e71-aa8b-43e9-8ca2-26684a458a77 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:05.405896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb76e71-aa8b-43e9-8ca2-26684a458a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:05.423803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-acb76e71-aa8b-43e9-8ca2-26684a458a77 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:05.424946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-acb76e71-aa8b-43e9-8ca2-26684a458a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:05.432590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:05.469765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5884721-4e60-4e86-af94-a0ab2219bc0b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:05.480315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5884721-4e60-4e86-af94-a0ab2219bc0b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:05.483825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0aedb1-3bce-4808-81f1-99c979214917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:05.484453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aedb1-3bce-4808-81f1-99c979214917 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:05.484734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aedb1-3bce-4808-81f1-99c979214917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:05.485009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0aedb1-3bce-4808-81f1-99c979214917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:05.493635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0aedb1-3bce-4808-81f1-99c979214917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:05.495041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0aedb1-3bce-4808-81f1-99c979214917 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:05.504265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 733/1470] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:15.418745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=11 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:07:15.500250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=17 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:07:38.612764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fbe96-2355-449f-a029-d5d43ea0b13c None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:38.613445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fbe96-2355-449f-a029-d5d43ea0b13c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:38.613445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fbe96-2355-449f-a029-d5d43ea0b13c None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:38.614939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 738/1471] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:38.620160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f901a-6fb5-4c52-b9ef-0f5e3f464f00 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:38.620656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f901a-6fb5-4c52-b9ef-0f5e3f464f00 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:38.621406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f901a-6fb5-4c52-b9ef-0f5e3f464f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:38.639753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d70f901a-6fb5-4c52-b9ef-0f5e3f464f00 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:38.640948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d70f901a-6fb5-4c52-b9ef-0f5e3f464f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:38.648780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 734/1472] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:38.696092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53506a4-f95c-4cf5-a727-25b95633c097 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:38.696385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53506a4-f95c-4cf5-a727-25b95633c097 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:38.696666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53506a4-f95c-4cf5-a727-25b95633c097 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:38.698078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 739/1473] 10.222.0.35 () {54 vars in 962 bytes} [Tue May 24 10:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:38.702950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e17d03-da0c-4986-a00d-5fc28d82ece5 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:38.703240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e17d03-da0c-4986-a00d-5fc28d82ece5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:38.703517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e17d03-da0c-4986-a00d-5fc28d82ece5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:38.722795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-32e17d03-da0c-4986-a00d-5fc28d82ece5 None None] MFA Rules not processed for user `9af9eae0fa5b437e96c1c732dfe4fa7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:38.724017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-32e17d03-da0c-4986-a00d-5fc28d82ece5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:38.731053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 735/1474] 10.222.0.35 () {58 vars in 1059 bytes} [Tue May 24 10:07:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:38.738405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53506a4-f95c-4cf5-a727-25b95633c097 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:38.749769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53506a4-f95c-4cf5-a727-25b95633c097 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:38.753337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde758a-371d-43fd-afcf-7dd24b00044d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:38.753978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde758a-371d-43fd-afcf-7dd24b00044d service glance] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:38.754259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde758a-371d-43fd-afcf-7dd24b00044d service glance] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:38.754535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde758a-371d-43fd-afcf-7dd24b00044d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:38.764409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde758a-371d-43fd-afcf-7dd24b00044d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:38.765926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde758a-371d-43fd-afcf-7dd24b00044d service glance] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:38.775507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 740/1475] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:41.083796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691d266-26fe-4c0e-ae34-5f1aef2dc627 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:41.084393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691d266-26fe-4c0e-ae34-5f1aef2dc627 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:41.084393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691d266-26fe-4c0e-ae34-5f1aef2dc627 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:41.085772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 736/1476] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:41.091645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4e243-9123-4f6b-aa18-d2c3a9054621 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:41.092104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4e243-9123-4f6b-aa18-d2c3a9054621 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:41.092219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4e243-9123-4f6b-aa18-d2c3a9054621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:41.110414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b6f4e243-9123-4f6b-aa18-d2c3a9054621 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:41.111591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b6f4e243-9123-4f6b-aa18-d2c3a9054621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:41.119476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 741/1477] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:41.143163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691d266-26fe-4c0e-ae34-5f1aef2dc627 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:41.157127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d691d266-26fe-4c0e-ae34-5f1aef2dc627 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:41.161913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac1b10-58c7-47dc-848e-58604469b903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:41.162569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac1b10-58c7-47dc-848e-58604469b903 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:41.162854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac1b10-58c7-47dc-848e-58604469b903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:41.163133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac1b10-58c7-47dc-848e-58604469b903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:41.163879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac1b10-58c7-47dc-848e-58604469b903 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:41.165727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-3dac1b10-58c7-47dc-848e-58604469b903 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:07:41.165727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:07:41.165727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:07:41.166876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac1b10-58c7-47dc-848e-58604469b903 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:41.186430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 737/1478] 10.222.0.35 () {56 vars in 1108 bytes} [Tue May 24 10:07:41 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:07:43.434309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c572cb-8f75-4105-919e-7114dc4797ce None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:43.434848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c572cb-8f75-4105-919e-7114dc4797ce None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:43.434848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c572cb-8f75-4105-919e-7114dc4797ce None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:43.436291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 742/1479] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:43.441620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427616d3-3134-487d-b083-ff5f60416b35 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:43.442040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427616d3-3134-487d-b083-ff5f60416b35 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:43.442183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427616d3-3134-487d-b083-ff5f60416b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:43.459933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-427616d3-3134-487d-b083-ff5f60416b35 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:43.461155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-427616d3-3134-487d-b083-ff5f60416b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:43.469893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 738/1480] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:43.513587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c572cb-8f75-4105-919e-7114dc4797ce None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:43.529026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c572cb-8f75-4105-919e-7114dc4797ce None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:43.534458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e2714-7184-4253-8b44-a34cf1c5fbfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:43.535126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2714-7184-4253-8b44-a34cf1c5fbfa service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:43.535409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2714-7184-4253-8b44-a34cf1c5fbfa service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:43.535945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e2714-7184-4253-8b44-a34cf1c5fbfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:43.544577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e2714-7184-4253-8b44-a34cf1c5fbfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:43.546212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e2714-7184-4253-8b44-a34cf1c5fbfa service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:43.555668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 743/1481] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:45.933693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8297f52-73dd-4699-852c-a721c86988d2 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:45.934268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8297f52-73dd-4699-852c-a721c86988d2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:45.934268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8297f52-73dd-4699-852c-a721c86988d2 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:45.935586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 739/1482] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:45.940933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec394-2d59-45c8-84c6-25a4db5250c8 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:45.941411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec394-2d59-45c8-84c6-25a4db5250c8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:45.941522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fec394-2d59-45c8-84c6-25a4db5250c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:45.959221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-10fec394-2d59-45c8-84c6-25a4db5250c8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:45.960423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-10fec394-2d59-45c8-84c6-25a4db5250c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:45.968831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 744/1483] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:46.003654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8297f52-73dd-4699-852c-a721c86988d2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:46.014636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8297f52-73dd-4699-852c-a721c86988d2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:46.018125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8e9b1-b2d8-45cc-b88e-a9bc60cf7577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:46.018748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e9b1-b2d8-45cc-b88e-a9bc60cf7577 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:46.019028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e9b1-b2d8-45cc-b88e-a9bc60cf7577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:46.019301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8e9b1-b2d8-45cc-b88e-a9bc60cf7577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:46.028808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e9b1-b2d8-45cc-b88e-a9bc60cf7577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:46.030254 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8e9b1-b2d8-45cc-b88e-a9bc60cf7577 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:46.039662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 740/1484] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:48.281690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a53c2-49fe-4964-9061-70e066d19b02 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:48.282251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a53c2-49fe-4964-9061-70e066d19b02 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:48.282251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771a53c2-49fe-4964-9061-70e066d19b02 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:48.283649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 745/1485] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 10:07:48.289086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfb394-01d2-4335-9ece-01b020aad131 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:48.289516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfb394-01d2-4335-9ece-01b020aad131 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:48.289688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bfb394-01d2-4335-9ece-01b020aad131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:48.311434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f7bfb394-01d2-4335-9ece-01b020aad131 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:48.312616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f7bfb394-01d2-4335-9ece-01b020aad131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:48.320343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 741/1486] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:48.358161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a53c2-49fe-4964-9061-70e066d19b02 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:48.368647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771a53c2-49fe-4964-9061-70e066d19b02 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:48.372222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af8045-b651-40c9-b7e5-d3ac5cd3ca3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:48.372858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af8045-b651-40c9-b7e5-d3ac5cd3ca3b service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:48.373142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af8045-b651-40c9-b7e5-d3ac5cd3ca3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:48.373451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af8045-b651-40c9-b7e5-d3ac5cd3ca3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:48.382181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af8045-b651-40c9-b7e5-d3ac5cd3ca3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:48.383615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af8045-b651-40c9-b7e5-d3ac5cd3ca3b service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:48.394488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 746/1487] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:50.763760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6636-9f86-4e6b-b62d-a1942aabb2d6 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:50.764339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6636-9f86-4e6b-b62d-a1942aabb2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:50.764768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6636-9f86-4e6b-b62d-a1942aabb2d6 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:50.767415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 742/1488] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:07:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:50.772965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cd9dd-eca5-448a-98f7-26fbe99d0636 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:50.773428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cd9dd-eca5-448a-98f7-26fbe99d0636 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:50.773553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cd9dd-eca5-448a-98f7-26fbe99d0636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:50.791467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b40cd9dd-eca5-448a-98f7-26fbe99d0636 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:50.792797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b40cd9dd-eca5-448a-98f7-26fbe99d0636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:50.800706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 747/1489] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:50.844534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6636-9f86-4e6b-b62d-a1942aabb2d6 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:50.857526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6636-9f86-4e6b-b62d-a1942aabb2d6 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:50.861434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fdc86-81e8-4db7-8dec-d17754c88933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:50.862076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fdc86-81e8-4db7-8dec-d17754c88933 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:50.862382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fdc86-81e8-4db7-8dec-d17754c88933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:50.862659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fdc86-81e8-4db7-8dec-d17754c88933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:50.873483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fdc86-81e8-4db7-8dec-d17754c88933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:50.875142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fdc86-81e8-4db7-8dec-d17754c88933 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:50.885122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 743/1490] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:07:53.583413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145301e-ec46-4c6a-b623-1ca6c7cc74c8 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:53.584007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145301e-ec46-4c6a-b623-1ca6c7cc74c8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:53.584007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145301e-ec46-4c6a-b623-1ca6c7cc74c8 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:53.585345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 748/1491] 10.222.0.35 () {54 vars in 926 bytes} [Tue May 24 10:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:07:53.590523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575bc0-2c73-45a7-ac64-e3002789fdfa None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:53.590974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575bc0-2c73-45a7-ac64-e3002789fdfa None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:53.591089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95575bc0-2c73-45a7-ac64-e3002789fdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:53.608932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-95575bc0-2c73-45a7-ac64-e3002789fdfa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:07:53.610154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-95575bc0-2c73-45a7-ac64-e3002789fdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:07:53.617910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 744/1492] 10.222.0.35 () {58 vars in 1023 bytes} [Tue May 24 10:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:07:53.656214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145301e-ec46-4c6a-b623-1ca6c7cc74c8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:07:53.667082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145301e-ec46-4c6a-b623-1ca6c7cc74c8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:07:53.671217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f080c57-36c3-46a3-92bb-380a3bb959d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:07:53.671850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f080c57-36c3-46a3-92bb-380a3bb959d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:07:53.672133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f080c57-36c3-46a3-92bb-380a3bb959d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:07:53.672406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f080c57-36c3-46a3-92bb-380a3bb959d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:07:53.681110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f080c57-36c3-46a3-92bb-380a3bb959d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:07:53.682574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f080c57-36c3-46a3-92bb-380a3bb959d5 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:07:53.692886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 749/1493] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:08:03.603346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=33 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:08:03.688429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=33 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:09:42.482145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5cb8e-3ca7-471d-be62-adcdd846ca12 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.482738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5cb8e-3ca7-471d-be62-adcdd846ca12 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.482738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5cb8e-3ca7-471d-be62-adcdd846ca12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:42.500436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-78b5cb8e-3ca7-471d-be62-adcdd846ca12 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:09:42.501667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-78b5cb8e-3ca7-471d-be62-adcdd846ca12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:09:42.509419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 745/1494] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:09:42.518628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a7e2-3b97-4b50-9d98-6a96b2259076 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.519142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a7e2-3b97-4b50-9d98-6a96b2259076 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.519247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6a7e2-3b97-4b50-9d98-6a96b2259076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:42.537359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-56b6a7e2-3b97-4b50-9d98-6a96b2259076 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:09:42.538540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-56b6a7e2-3b97-4b50-9d98-6a96b2259076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:09:42.547188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 750/1495] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:09:42.551799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5cb8e-3ca7-471d-be62-adcdd846ca12 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:42.561547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5cb8e-3ca7-471d-be62-adcdd846ca12 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:42.565912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:42.566562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.566844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.567121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:42.567840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:09:42.568144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:42.569152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:42.569152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:09:42.569152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:09:42.570049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:42.581444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 746/1496] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:09:42.589349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6a7e2-3b97-4b50-9d98-6a96b2259076 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:42.599094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6a7e2-3b97-4b50-9d98-6a96b2259076 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:42.603618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:42.604264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.604550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.604826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:42.606290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:42.608050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING py.warnings [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:42.608050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: warnings.warn(msg) May 24 10:09:42.608050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: May 24 10:09:42.609225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:42.733946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 751/1497] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:09:42 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:09:42.737833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:42.747942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63799e-f21e-43c4-b29d-68853400b902 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:42.803518 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:42.804184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.804619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.804918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:42.805648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:42.806665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:42.806665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:09:42.806665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:09:42.807881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:42.892130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 747/1498] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:09:42 2022] POST /identity/v3/users => generated 290 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:09:42.896374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:42.907267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2d8e5-c61e-4127-bd00-2f7ca3aa07e8 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:42.911851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:42.912510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.912794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.913070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:42.913830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:42.915126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING py.warnings [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:42.915126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: warnings.warn(msg) May 24 10:09:42.915126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: May 24 10:09:42.915996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:42.932457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 752/1499] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:09:42.937548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:42.947525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266800fb-9543-41f4-9786-dab32b0daea1 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:42.953108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccae1e6-f3c9-4a1e-ac6d-95de75a2e3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:42.953811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccae1e6-f3c9-4a1e-ac6d-95de75a2e3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.954098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccae1e6-f3c9-4a1e-ac6d-95de75a2e3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.954380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccae1e6-f3c9-4a1e-ac6d-95de75a2e3f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:42.955107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccae1e6-f3c9-4a1e-ac6d-95de75a2e3f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:42.956123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-eccae1e6-f3c9-4a1e-ac6d-95de75a2e3f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:42.956123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:09:42.956123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:09:42.957040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccae1e6-f3c9-4a1e-ac6d-95de75a2e3f6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:42.971184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 748/1500] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:09:42.975289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:42.985205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ae05-c85a-4ef0-829c-f860357cc0fe admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:42.989685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31271624-0dd9-4979-97f7-f57826ed8ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:42.990556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271624-0dd9-4979-97f7-f57826ed8ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:42.990840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271624-0dd9-4979-97f7-f57826ed8ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:42.991115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31271624-0dd9-4979-97f7-f57826ed8ad0 admin admin] PATH_INFO: `/v3/projects/e2dadf42741042589b545cd68c01a522/users/a1f0f58a7f0b4a5f95727d5f959b29bd/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.005562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31271624-0dd9-4979-97f7-f57826ed8ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2dadf42741042589b545cd68c01a522, user_id=a1f0f58a7f0b4a5f95727d5f959b29bd, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:43.006601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING py.warnings [None req-31271624-0dd9-4979-97f7-f57826ed8ad0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:43.006601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: warnings.warn(msg) May 24 10:09:43.006601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: May 24 10:09:43.008499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31271624-0dd9-4979-97f7-f57826ed8ad0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:43.068364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 753/1501] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:09:42 2022] PUT /identity/v3/projects/e2dadf42741042589b545cd68c01a522/users/a1f0f58a7f0b4a5f95727d5f959b29bd/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:09:43.080568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ed00-29af-4f58-a6ea-ecf2f7f7607d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:43.080860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ed00-29af-4f58-a6ea-ecf2f7f7607d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:43.081144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756ed00-29af-4f58-a6ea-ecf2f7f7607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.107794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4756ed00-29af-4f58-a6ea-ecf2f7f7607d None None] MFA Rules not processed for user `a1f0f58a7f0b4a5f95727d5f959b29bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:09:43.108999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4756ed00-29af-4f58-a6ea-ecf2f7f7607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:09:43.173281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 749/1502] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:09:43.187427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee958ef5-c751-4874-b919-82d3aa3701b0 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:43.187718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee958ef5-c751-4874-b919-82d3aa3701b0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:43.188005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee958ef5-c751-4874-b919-82d3aa3701b0 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.189404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 754/1503] 10.222.0.35 () {54 vars in 963 bytes} [Tue May 24 10:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:09:43.194343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c9728-773d-40b4-9731-6ca30582d5c0 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:43.194687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c9728-773d-40b4-9731-6ca30582d5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:43.194971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c9728-773d-40b4-9731-6ca30582d5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.213285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4f8c9728-773d-40b4-9731-6ca30582d5c0 None None] MFA Rules not processed for user `aaba1a8ba921489e96f9fa4acecffb74`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:09:43.214497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4f8c9728-773d-40b4-9731-6ca30582d5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:09:43.257299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:09:43.264868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 750/1504] 10.222.0.35 () {58 vars in 1060 bytes} [Tue May 24 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:09:43.272570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee958ef5-c751-4874-b919-82d3aa3701b0 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:43.283896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee958ef5-c751-4874-b919-82d3aa3701b0 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:43.287654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:43.288301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:43.288593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:43.288870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.298558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:43.299994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:43.309000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 755/1505] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:09:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:09:43.351934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf1030-8936-4f06-b5c4-c4e2bbfb908e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:43.352229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf1030-8936-4f06-b5c4-c4e2bbfb908e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:43.352512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf1030-8936-4f06-b5c4-c4e2bbfb908e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.353971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 751/1506] 10.222.0.35 () {52 vars in 829 bytes} [Tue May 24 10:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:09:43.358222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:43.419195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f5ff51-f30f-43d9-ae42-77a65c6ae57d service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:43.423776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:43.424650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:43.424932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:43.425205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] PATH_INFO: `/v3/users/a1f0f58a7f0b4a5f95727d5f959b29bd` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.426410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f0f58a7f0b4a5f95727d5f959b29bd)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:43.427455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING py.warnings [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:43.427455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: warnings.warn(msg) May 24 10:09:43.427455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: May 24 10:09:43.428642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:43.647029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f0f58a7f0b4a5f95727d5f959b29bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:43.660019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f0f58a7f0b4a5f95727d5f959b29bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:43.660778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f0f58a7f0b4a5f95727d5f959b29bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:43.680703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f0f58a7f0b4a5f95727d5f959b29bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:43.696090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:09:43.937390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f0f58a7f0b4a5f95727d5f959b29bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:43.943769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 756/1507] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:09:43 2022] DELETE /identity/v3/users/a1f0f58a7f0b4a5f95727d5f959b29bd => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:09:43.948413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:43.948912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:43.949006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:43.969407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:09:43.970649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:09:44.027554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 752/1508] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:09:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:09:44.037116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:44.038644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:09:44.095915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd746d9b-f7a1-4c63-ba9c-46f67b11cfa4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:44.099546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143e15d-6599-4e2d-b499-91a9a02cd743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:44.100182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143e15d-6599-4e2d-b499-91a9a02cd743 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:44.100468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143e15d-6599-4e2d-b499-91a9a02cd743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:44.100744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143e15d-6599-4e2d-b499-91a9a02cd743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:44.110064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143e15d-6599-4e2d-b499-91a9a02cd743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:44.111510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143e15d-6599-4e2d-b499-91a9a02cd743 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:44.120839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 757/1509] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:09:44.585685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:09:44.587128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:09:44.600251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20fadc-148e-4632-bd4d-9fec3ccabbfe None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:09:44.606376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:09:44.614398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:09:44.614398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:09:44.614398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] PATH_INFO: `/v3/projects/e2dadf42741042589b545cd68c01a522` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:09:44.614398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2dadf42741042589b545cd68c01a522)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:09:44.614398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:09:44.614398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:09:44.614398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:09:44.615298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:09:44.763818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2dadf42741042589b545cd68c01a522', 'request_id': 'req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:44.834520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2dadf42741042589b545cd68c01a522', 'request_id': 'req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:44.841604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2dadf42741042589b545cd68c01a522', 'request_id': 'req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:44.860405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] The token cache is being invalidate because project e2dadf42741042589b545cd68c01a522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:09:44.860691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:44.861393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53e3e4f-38d8-4c88-8a82-f7b35e6acbda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:09:44.863036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 753/1510] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:09:44 2022] DELETE /identity/v3/projects/e2dadf42741042589b545cd68c01a522 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:09:54.116670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:09:54.857495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:11:22.755382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e2675-be9a-4167-b1c6-23a5a9e3a72a None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:22.756000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e2675-be9a-4167-b1c6-23a5a9e3a72a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:22.756000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e2675-be9a-4167-b1c6-23a5a9e3a72a None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:22.757379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:22.763608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5c3c7-3fb8-4045-ac34-dfeb97d8d61d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:22.764042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5c3c7-3fb8-4045-ac34-dfeb97d8d61d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:22.764165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b5c3c7-3fb8-4045-ac34-dfeb97d8d61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:22.784046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-94b5c3c7-3fb8-4045-ac34-dfeb97d8d61d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:22.785344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-94b5c3c7-3fb8-4045-ac34-dfeb97d8d61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:22.848080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:23.051428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e2675-be9a-4167-b1c6-23a5a9e3a72a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:23.053149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-301e2675-be9a-4167-b1c6-23a5a9e3a72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:23.127343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e2675-be9a-4167-b1c6-23a5a9e3a72a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:23.133102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eccab26-cc66-40df-a8cc-34b6ccdb2d6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:23.133784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccab26-cc66-40df-a8cc-34b6ccdb2d6b service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:23.134072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccab26-cc66-40df-a8cc-34b6ccdb2d6b service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:23.134351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eccab26-cc66-40df-a8cc-34b6ccdb2d6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:23.143279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccab26-cc66-40df-a8cc-34b6ccdb2d6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:23.145188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eccab26-cc66-40df-a8cc-34b6ccdb2d6b service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:23.155098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 759/1513] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:23.957546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afa60b-ad8f-4628-b1ae-5dd51d9fc3e3 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:23.958141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afa60b-ad8f-4628-b1ae-5dd51d9fc3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:23.958141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afa60b-ad8f-4628-b1ae-5dd51d9fc3e3 None None] PATH_INFO: `/v3` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:23.959420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 755/1514] 10.222.0.16 () {54 vars in 925 bytes} [Tue May 24 10:11:23 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:11:23.973107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e8a8a-d999-492f-b3e7-280a23f4e292 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:23.973577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e8a8a-d999-492f-b3e7-280a23f4e292 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:23.973708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562e8a8a-d999-492f-b3e7-280a23f4e292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:23.992187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-562e8a8a-d999-492f-b3e7-280a23f4e292 None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:23.993411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-562e8a8a-d999-492f-b3e7-280a23f4e292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:24.052832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc280-d25d-47e0-bfc0-6980c15f8f9e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:24.053033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc280-d25d-47e0-bfc0-6980c15f8f9e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:24.053309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cc280-d25d-47e0-bfc0-6980c15f8f9e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:24.054829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 756/1515] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 24 10:11:24.059284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9beb81-f26f-4f5e-a38b-e964284abe66 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:24.059432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 760/1516] 10.222.0.16 () {58 vars in 1015 bytes} [Tue May 24 10:11:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:24.059555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9beb81-f26f-4f5e-a38b-e964284abe66 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:24.059837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9beb81-f26f-4f5e-a38b-e964284abe66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:24.077437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0d9beb81-f26f-4f5e-a38b-e964284abe66 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:24.078626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0d9beb81-f26f-4f5e-a38b-e964284abe66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:24.087222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:11:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:11:24.244010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e8a8a-d999-492f-b3e7-280a23f4e292 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:24.256039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562e8a8a-d999-492f-b3e7-280a23f4e292 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:24.261282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67bed84-4c56-4cc4-8f94-99e4a3fe653e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:24.261954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bed84-4c56-4cc4-8f94-99e4a3fe653e service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:24.262236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bed84-4c56-4cc4-8f94-99e4a3fe653e service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:24.262510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67bed84-4c56-4cc4-8f94-99e4a3fe653e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:24.271504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67bed84-4c56-4cc4-8f94-99e4a3fe653e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:24.273101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67bed84-4c56-4cc4-8f94-99e4a3fe653e service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:24.282604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 761/1518] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:28.272519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d6dae-4ff3-449b-81f7-87a7d8f1282e None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:28.273065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d6dae-4ff3-449b-81f7-87a7d8f1282e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:28.273181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d6dae-4ff3-449b-81f7-87a7d8f1282e None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:28.274490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 758/1519] 10.222.0.35 () {54 vars in 960 bytes} [Tue May 24 10:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:28.280623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cabe80-155e-485c-952f-17ec779a1ffe None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:28.281060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cabe80-155e-485c-952f-17ec779a1ffe None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:28.281223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cabe80-155e-485c-952f-17ec779a1ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:28.300845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e8cabe80-155e-485c-952f-17ec779a1ffe None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:28.302070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e8cabe80-155e-485c-952f-17ec779a1ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:28.310519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 762/1520] 10.222.0.35 () {58 vars in 1057 bytes} [Tue May 24 10:11:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:28.318129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d6dae-4ff3-449b-81f7-87a7d8f1282e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:28.331824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d6dae-4ff3-449b-81f7-87a7d8f1282e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:28.336641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483049a-329c-41bb-9a18-bb61f589a39f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:28.337260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483049a-329c-41bb-9a18-bb61f589a39f service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:28.337571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483049a-329c-41bb-9a18-bb61f589a39f service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:28.337847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483049a-329c-41bb-9a18-bb61f589a39f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:28.346706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483049a-329c-41bb-9a18-bb61f589a39f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:28.348339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483049a-329c-41bb-9a18-bb61f589a39f service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:28.358103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 759/1521] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:35.542845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e942-e2b4-4b86-8f15-d0b4a0d846cf None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:35.543710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e942-e2b4-4b86-8f15-d0b4a0d846cf None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:35.543710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e942-e2b4-4b86-8f15-d0b4a0d846cf None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:35.545058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:35.549813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba960c-e415-4ade-9bab-b8269059670e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:35.550280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba960c-e415-4ade-9bab-b8269059670e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:35.550386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba960c-e415-4ade-9bab-b8269059670e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:35.568412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-63ba960c-e415-4ade-9bab-b8269059670e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:35.569629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-63ba960c-e415-4ade-9bab-b8269059670e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:35.577242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:35.732574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744e942-e2b4-4b86-8f15-d0b4a0d846cf None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:35.747428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744e942-e2b4-4b86-8f15-d0b4a0d846cf None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:35.752886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c9848-c797-4a84-a8ce-a81c47f093ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:35.753548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c9848-c797-4a84-a8ce-a81c47f093ec service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:35.753828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c9848-c797-4a84-a8ce-a81c47f093ec service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:35.754101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c9848-c797-4a84-a8ce-a81c47f093ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:35.762102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c9848-c797-4a84-a8ce-a81c47f093ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:35.763699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c9848-c797-4a84-a8ce-a81c47f093ec service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:35.773065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 764/1524] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:36.660367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5b39-ce20-4fdd-b0b1-67181296e90b None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:36.660961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5b39-ce20-4fdd-b0b1-67181296e90b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:36.660961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae5b39-ce20-4fdd-b0b1-67181296e90b None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:36.662356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue May 24 10:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:36.667079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86e4d4-9a7b-4eb5-b163-7852a9017a25 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:36.667516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86e4d4-9a7b-4eb5-b163-7852a9017a25 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:36.667628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86e4d4-9a7b-4eb5-b163-7852a9017a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:36.686376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ab86e4d4-9a7b-4eb5-b163-7852a9017a25 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:36.687571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ab86e4d4-9a7b-4eb5-b163-7852a9017a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:36.695699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue May 24 10:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:36.848126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5b39-ce20-4fdd-b0b1-67181296e90b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:36.862359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae5b39-ce20-4fdd-b0b1-67181296e90b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:36.867558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef212a-5381-499f-bed8-e55092c18f79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:36.868182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef212a-5381-499f-bed8-e55092c18f79 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:36.868462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef212a-5381-499f-bed8-e55092c18f79 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:36.868736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef212a-5381-499f-bed8-e55092c18f79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:36.876602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef212a-5381-499f-bed8-e55092c18f79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:36.878372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef212a-5381-499f-bed8-e55092c18f79 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:36.887468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 762/1527] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:41.180700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ddb5a-5faa-4161-9dee-71070e72ce12 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:41.181181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ddb5a-5faa-4161-9dee-71070e72ce12 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:41.181325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ddb5a-5faa-4161-9dee-71070e72ce12 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:41.182687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:41.187875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b10b99-8265-4ec6-8192-bb742c60e15b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:41.188281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b10b99-8265-4ec6-8192-bb742c60e15b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:41.188435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b10b99-8265-4ec6-8192-bb742c60e15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:41.206750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-24b10b99-8265-4ec6-8192-bb742c60e15b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:41.207959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-24b10b99-8265-4ec6-8192-bb742c60e15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:41.215960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:41.252308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ddb5a-5faa-4161-9dee-71070e72ce12 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:41.254051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ea8ddb5a-5faa-4161-9dee-71070e72ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:41.299490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:11:41.310969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ddb5a-5faa-4161-9dee-71070e72ce12 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:41.314479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e3055-b24b-4768-b982-501144d7e657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:41.315209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e3055-b24b-4768-b982-501144d7e657 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:41.315495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e3055-b24b-4768-b982-501144d7e657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:41.315773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e3055-b24b-4768-b982-501144d7e657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:41.324650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e3055-b24b-4768-b982-501144d7e657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:41.326118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e3055-b24b-4768-b982-501144d7e657 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:41.336200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 767/1530] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:43.543547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf04a1-8030-4964-8277-34561818bfe8 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:43.544355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf04a1-8030-4964-8277-34561818bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:43.544355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf04a1-8030-4964-8277-34561818bfe8 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:43.545759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:43.552755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76270f7c-c8f5-4930-b69c-832eb0b0e17d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:43.553218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76270f7c-c8f5-4930-b69c-832eb0b0e17d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:43.553355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76270f7c-c8f5-4930-b69c-832eb0b0e17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:43.574117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-76270f7c-c8f5-4930-b69c-832eb0b0e17d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:43.575318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-76270f7c-c8f5-4930-b69c-832eb0b0e17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:43.583122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:43.618622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf04a1-8030-4964-8277-34561818bfe8 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:43.629955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf04a1-8030-4964-8277-34561818bfe8 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:43.633519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ab372-bb89-486a-aa84-d753cba09793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:43.634148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab372-bb89-486a-aa84-d753cba09793 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:43.634431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab372-bb89-486a-aa84-d753cba09793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:43.634707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ab372-bb89-486a-aa84-d753cba09793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:43.643518 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ab372-bb89-486a-aa84-d753cba09793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:43.644936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ab372-bb89-486a-aa84-d753cba09793 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:43.654129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 765/1533] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:47.257190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a5960-8bfe-493a-a1eb-5b12cdd896d9 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:47.257740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a5960-8bfe-493a-a1eb-5b12cdd896d9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:47.257863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a5960-8bfe-493a-a1eb-5b12cdd896d9 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:47.259175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:47.264354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e179c0-6333-4495-bf3f-f2a936d23297 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:47.264782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e179c0-6333-4495-bf3f-f2a936d23297 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:47.264943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e179c0-6333-4495-bf3f-f2a936d23297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:47.271920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=55 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:11:47.284164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-64e179c0-6333-4495-bf3f-f2a936d23297 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:47.285365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-64e179c0-6333-4495-bf3f-f2a936d23297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:47.293149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:47.565248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a5960-8bfe-493a-a1eb-5b12cdd896d9 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:47.567045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b46a5960-8bfe-493a-a1eb-5b12cdd896d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:47.624368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a5960-8bfe-493a-a1eb-5b12cdd896d9 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:47.628030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d85a0db-4a47-42d1-996b-6ee133f40f0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:47.628677 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85a0db-4a47-42d1-996b-6ee133f40f0a service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:47.628960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85a0db-4a47-42d1-996b-6ee133f40f0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:47.629236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d85a0db-4a47-42d1-996b-6ee133f40f0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:47.638064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85a0db-4a47-42d1-996b-6ee133f40f0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:47.639504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d85a0db-4a47-42d1-996b-6ee133f40f0a service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:47.648842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 770/1536] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:49.901719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706821c-205a-472b-aff2-fada89b2c5a6 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:49.902284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706821c-205a-472b-aff2-fada89b2c5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:49.902284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4706821c-205a-472b-aff2-fada89b2c5a6 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:49.903632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue May 24 10:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:11:49.909097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5116e6-906e-4a74-8d60-f1a91913abee None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:49.909529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5116e6-906e-4a74-8d60-f1a91913abee None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:49.909623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5116e6-906e-4a74-8d60-f1a91913abee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:49.927542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2e5116e6-906e-4a74-8d60-f1a91913abee None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:11:49.928715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2e5116e6-906e-4a74-8d60-f1a91913abee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:49.936359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue May 24 10:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:11:50.216094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706821c-205a-472b-aff2-fada89b2c5a6 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:11:50.217827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4706821c-205a-472b-aff2-fada89b2c5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:11:50.230603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4706821c-205a-472b-aff2-fada89b2c5a6 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:11:50.234072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c92ac4-6b24-492b-8116-ec592d9c095c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:11:50.234703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92ac4-6b24-492b-8116-ec592d9c095c service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:11:50.234988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92ac4-6b24-492b-8116-ec592d9c095c service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:11:50.235264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c92ac4-6b24-492b-8116-ec592d9c095c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:11:50.245048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c92ac4-6b24-492b-8116-ec592d9c095c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:11:50.246524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c92ac4-6b24-492b-8116-ec592d9c095c service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:11:50.256395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 768/1539] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:11:59.922427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=39 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:12:00.251694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=8 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:22.886801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825982-3965-4a06-8db9-faf1bdf758c4 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:22.887439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825982-3965-4a06-8db9-faf1bdf758c4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:22.887439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f825982-3965-4a06-8db9-faf1bdf758c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:22.915311 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-3f825982-3965-4a06-8db9-faf1bdf758c4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:22.916519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3f825982-3965-4a06-8db9-faf1bdf758c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:22.929697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 772/1540] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:22.939618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc1631-36e6-4c1c-9221-7a15f11ce7f1 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:22.940225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc1631-36e6-4c1c-9221-7a15f11ce7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:22.940225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc1631-36e6-4c1c-9221-7a15f11ce7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:22.951319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c218d-bad3-471a-9bb3-9b50bd1b10c9 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:22.951676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c218d-bad3-471a-9bb3-9b50bd1b10c9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:22.951892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c218d-bad3-471a-9bb3-9b50bd1b10c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:22.971594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-fcbc1631-36e6-4c1c-9221-7a15f11ce7f1 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:22.973578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fcbc1631-36e6-4c1c-9221-7a15f11ce7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:22.976630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1f3c218d-bad3-471a-9bb3-9b50bd1b10c9 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:22.978062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1f3c218d-bad3-471a-9bb3-9b50bd1b10c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:22.991404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 773/1541] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:13:22.993078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 769/1542] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:22.994461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff596cff-260c-453e-9ac5-b8f28ea9d062 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:22.994751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff596cff-260c-453e-9ac5-b8f28ea9d062 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:22.995030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff596cff-260c-453e-9ac5-b8f28ea9d062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:22.997911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2d382-66f6-4b3f-947d-f6eaf0b05dd2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:22.998383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2d382-66f6-4b3f-947d-f6eaf0b05dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:22.998671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b2d382-66f6-4b3f-947d-f6eaf0b05dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.037713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ff596cff-260c-453e-9ac5-b8f28ea9d062 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.040177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ff596cff-260c-453e-9ac5-b8f28ea9d062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.040622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-36b2d382-66f6-4b3f-947d-f6eaf0b05dd2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.042041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-36b2d382-66f6-4b3f-947d-f6eaf0b05dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.058964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 770/1543] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.059731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 774/1544] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.060827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2d382-66f6-4b3f-947d-f6eaf0b05dd2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.063020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492474b2-fdb9-41b0-b409-3e38d54f5263 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.067348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492474b2-fdb9-41b0-b409-3e38d54f5263 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.068048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492474b2-fdb9-41b0-b409-3e38d54f5263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.082478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b2d382-66f6-4b3f-947d-f6eaf0b05dd2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.094435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.095778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.096092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.096368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.097145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.097783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.098873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING py.warnings [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 24 10:13:23.098873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: warnings.warn(msg) May 24 10:13:23.098873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: May 24 10:13:23.099752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1b2ce4-f013-4841-b3d6-3891709798ff admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.101652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-492474b2-fdb9-41b0-b409-3e38d54f5263 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.102935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-492474b2-fdb9-41b0-b409-3e38d54f5263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.114354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 771/1545] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.114354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab2658-fc98-4c7e-bc6f-33abb325ad53 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.114354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab2658-fc98-4c7e-bc6f-33abb325ad53 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.114354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab2658-fc98-4c7e-bc6f-33abb325ad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.118727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 775/1546] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.119304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de048f07-111b-47ac-8d39-e66cf3643f4b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.119596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de048f07-111b-47ac-8d39-e66cf3643f4b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.119873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de048f07-111b-47ac-8d39-e66cf3643f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.137005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-39ab2658-fc98-4c7e-bc6f-33abb325ad53 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.138274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-39ab2658-fc98-4c7e-bc6f-33abb325ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.140172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-de048f07-111b-47ac-8d39-e66cf3643f4b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.141909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-de048f07-111b-47ac-8d39-e66cf3643f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.149932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 776/1547] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.150636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 772/1548] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.151925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464f513-2651-4fc0-b906-f70c5a1cd925 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.152215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464f513-2651-4fc0-b906-f70c5a1cd925 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.152495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464f513-2651-4fc0-b906-f70c5a1cd925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.154516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b31faf-3646-48b8-bbbd-fb89bfa2b702 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.154809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b31faf-3646-48b8-bbbd-fb89bfa2b702 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.155090 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b31faf-3646-48b8-bbbd-fb89bfa2b702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.173239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-3464f513-2651-4fc0-b906-f70c5a1cd925 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.174472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3464f513-2651-4fc0-b906-f70c5a1cd925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.174881 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c4b31faf-3646-48b8-bbbd-fb89bfa2b702 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.176719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c4b31faf-3646-48b8-bbbd-fb89bfa2b702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.183092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 773/1549] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.184191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ff43-88cf-4536-80dd-6a6ef65fcdbc None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.184483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ff43-88cf-4536-80dd-6a6ef65fcdbc None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.184581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 777/1550] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.184762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874ff43-88cf-4536-80dd-6a6ef65fcdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.185423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b31faf-3646-48b8-bbbd-fb89bfa2b702 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.196259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b31faf-3646-48b8-bbbd-fb89bfa2b702 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.200887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d636f7-868d-4d80-8eb5-945cf605e9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.201572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d636f7-868d-4d80-8eb5-945cf605e9d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.201859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d636f7-868d-4d80-8eb5-945cf605e9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.202211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d636f7-868d-4d80-8eb5-945cf605e9d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.202942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d636f7-868d-4d80-8eb5-945cf605e9d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.203254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d636f7-868d-4d80-8eb5-945cf605e9d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.204953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d636f7-868d-4d80-8eb5-945cf605e9d9 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.206039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5874ff43-88cf-4536-80dd-6a6ef65fcdbc None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.207243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5874ff43-88cf-4536-80dd-6a6ef65fcdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.215263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 774/1551] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.216066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874ff43-88cf-4536-80dd-6a6ef65fcdbc None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.222822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 778/1552] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.224128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01695f8c-9066-4d5f-b359-f25bb6adb7c9 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.224416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01695f8c-9066-4d5f-b359-f25bb6adb7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.224729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01695f8c-9066-4d5f-b359-f25bb6adb7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.228871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874ff43-88cf-4536-80dd-6a6ef65fcdbc None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.234336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.235042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.235372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.235695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.236410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.237522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:13:23.237522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:13:23.237522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:13:23.239290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.243304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-01695f8c-9066-4d5f-b359-f25bb6adb7c9 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.244480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-01695f8c-9066-4d5f-b359-f25bb6adb7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.252166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 779/1553] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.253118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01695f8c-9066-4d5f-b359-f25bb6adb7c9 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.263568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01695f8c-9066-4d5f-b359-f25bb6adb7c9 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.268539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94591252-7ffb-4f7a-a6aa-27049ef0a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.268955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94591252-7ffb-4f7a-a6aa-27049ef0a039 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.269978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94591252-7ffb-4f7a-a6aa-27049ef0a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.270249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94591252-7ffb-4f7a-a6aa-27049ef0a039 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.271015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94591252-7ffb-4f7a-a6aa-27049ef0a039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.271359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94591252-7ffb-4f7a-a6aa-27049ef0a039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.273006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94591252-7ffb-4f7a-a6aa-27049ef0a039 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.284779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 780/1554] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.292789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cb219-b94a-475d-b824-60dc432c3b41 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.292789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cb219-b94a-475d-b824-60dc432c3b41 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.292789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4cb219-b94a-475d-b824-60dc432c3b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.304806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ff4cb219-b94a-475d-b824-60dc432c3b41 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.306014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ff4cb219-b94a-475d-b824-60dc432c3b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.313949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 781/1555] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.315121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac783f-b2c5-4ad7-9f09-9528202e9365 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.315404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac783f-b2c5-4ad7-9f09-9528202e9365 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.315706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac783f-b2c5-4ad7-9f09-9528202e9365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.327296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 775/1556] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/projects => generated 403 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:23.330146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.334070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ddac783f-b2c5-4ad7-9f09-9528202e9365 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.335256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ddac783f-b2c5-4ad7-9f09-9528202e9365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.394804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c23e1-c9a3-40ff-8d08-bc1a19cd31ca admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.396508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 782/1557] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.397257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.397950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.398192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1095a8-b389-4475-babd-13a07b81db76 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.398298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.398483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1095a8-b389-4475-babd-13a07b81db76 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.398581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.398782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1095a8-b389-4475-babd-13a07b81db76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.399218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.399530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.401184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.413752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 776/1558] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.415317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.418843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ab1095a8-b389-4475-babd-13a07b81db76 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.420106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ab1095a8-b389-4475-babd-13a07b81db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.427252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e034fe-e9c7-4dcb-b17b-d2d550ee4ac7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.434111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 783/1559] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.434111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1530f-9721-4d2b-821a-2591b126609c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.434111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1530f-9721-4d2b-821a-2591b126609c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.434111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1530f-9721-4d2b-821a-2591b126609c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.434783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.434783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.434783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.434783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.435386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.437106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.580580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0ef1530f-9721-4d2b-821a-2591b126609c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:23.581896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0ef1530f-9721-4d2b-821a-2591b126609c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:23.590987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 784/1560] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:23.595300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1530f-9721-4d2b-821a-2591b126609c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.595420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 777/1561] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/projects => generated 379 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:23.595420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.605678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1530f-9721-4d2b-821a-2591b126609c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.657841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c630cf-6e3c-4f56-890a-c44aedabef61 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.663345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.664009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.664291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.664599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.665092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.665424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.665729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.665836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.666102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.666381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.667132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.667356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.669366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.674660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:23.680944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 778/1562] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.682016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.688347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:23.693838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f68c8df-3d44-4759-9e70-ddd07f8a4b5d admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.697750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.698414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.698754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.699063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.699830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.701927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.798992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 785/1563] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:23.799922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.815145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 779/1564] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/users => generated 302 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:23.816305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.826397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52771b56-0a81-466c-b4c7-e4edd129d0cf admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.865249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befcc59c-14dc-4c77-a439-4afd654d781c admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.869991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.870741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.871023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.871297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.872028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.872334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.873996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.885073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.893485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 786/1565] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.894033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.894033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.894033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.894792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.894869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.894869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.899520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b3389-9729-4a74-9fc0-568d8f64eb39 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.904119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.905920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.905920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.905920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.906214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.906513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.908231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.921207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 787/1566] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.922068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.932157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a72a2b-b3fb-495a-9879-b3a69a2a4497 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.936813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.937476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.937755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.938025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.938734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:23.939068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.940661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.952325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 788/1567] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:23.953079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:23.962866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1e7d3-7347-4990-8f89-1731e10ffcd4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:23.967388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:23.968015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:23.968297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:23.968574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:23.969244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:23.970303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING py.warnings [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:13:23.970303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: warnings.warn(msg) May 24 10:13:23.970303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: May 24 10:13:23.971649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:23.982366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 780/1568] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/projects => generated 399 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:23.982366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.043725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920756e-7306-4757-b2bb-d81805e1cbac admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.048877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.049577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.049862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.050140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.050908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.052809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.086086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 789/1569] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/users => generated 290 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.091187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.096512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f78df-53d8-46b7-b270-6979dd83a2d4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.101814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.102458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.102740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.103020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.103691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.105640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.193496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 781/1570] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:23 2022] POST /identity/v3/projects => generated 377 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.194458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.226978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 790/1571] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.226978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.236240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b4330-ffac-4dd0-a162-6197fed1ef43 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.254315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:24.258274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1e931-10ee-404b-9dd6-9e25ef9f5640 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.263906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.264575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.264863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.265174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.265966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.267621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.278606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 782/1572] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:24.279510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.290513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f68e1d-2d64-4128-976e-502cbe08b1d8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.293451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.294426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.294713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.295163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.295274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.295852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.295950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.296151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.296431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.297109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.299714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.299857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.452449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 783/1573] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.453454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.514616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd116b-b7fa-4b20-b281-4f0c77d03ce4 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.518861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 791/1574] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/projects => generated 393 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.529027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.529027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.529027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.529027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.529464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.529558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.529651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.529738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:24.585947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3401ea5-b8cb-409a-88f0-2fee17ea1398 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.590643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.591365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.591649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.592015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.592710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.594661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.668741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 784/1575] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.669698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.723183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 792/1576] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.723183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.724047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c801dcc-d093-4ca4-8c22-db4fbb884166 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.728574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.729223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.729544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.729821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.730503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.732139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92aed6-8e71-40cf-95f4-dde448c95986 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.732443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.736799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.737492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.737784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.738665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.739561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.741151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.751103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 785/1577] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:24.751932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.762471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266864e-abd1-4a0f-86b2-3ec5cab8d882 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.767344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.767971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.768255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.768530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.769315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.770941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.780803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 786/1578] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:24.781636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.790944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff83334-8702-4453-8724-0ea11d108a18 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.795578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.796233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.796513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.796827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.797870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.799478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.810480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 787/1579] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:24.811256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.821040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ab166e-4a29-4582-8ce2-eb01f7bcca89 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.825617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.826253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.826534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.826810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.827469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.829323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.851310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 793/1580] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.852186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.861487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bb3d47-77c1-4e29-a42b-afba54f5203e admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.866668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.867309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.867590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.867898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.868572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.870560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.941696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 788/1581] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/users => generated 287 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.942575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.952298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ef6ac-c896-4f47-82f9-950092eef107 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.957584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.958255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.958536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.958810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.959495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:24.961485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:24.971443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:24.984041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 794/1582] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:24.984041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:24.991651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348804a-816e-4ed6-81f9-5c4854cc2ead admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:24.996511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:24.997162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:24.997482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:24.997759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:24.998475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.000066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.011511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 795/1583] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.015744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.022787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27580e5-af4b-4e07-849f-b9ac65306d53 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.027334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.027963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.028244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.028519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.029223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.030811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.041082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 796/1584] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.042083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.052312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25621b87-1353-47bf-b113-74a47b61b412 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.057071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.069659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.069659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.069659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.069659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.069659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.071265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 797/1585] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.072006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.081245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fb1de-40f3-4e57-aa53-cc45c2a5beb2 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.082296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 789/1586] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:24 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:25.083122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.088781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.089771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.090102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.090383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] PATH_INFO: `/v3/projects/647c3ee6b60f4edd92dc08ff7a201d1a/users/927cd6ea4d854f48ad2d92073dd84c55/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.094664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273e3dad-5d47-4a80-952a-36e849b58196 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.099995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.107795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.107795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.107795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.107795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.107795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.108273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647c3ee6b60f4edd92dc08ff7a201d1a, user_id=927cd6ea4d854f48ad2d92073dd84c55, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.109506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.115281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 790/1587] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.116317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.125884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17cd0d-7f5c-41a3-8c2e-c2bd861208a2 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.130736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.131368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.131655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.131935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.132646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.135457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.146758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 791/1588] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.147596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.161789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bba17f1-8887-4865-a69f-ed6f9022088d admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.168304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.169053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.169402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.169708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.170555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.172537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.183454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 792/1589] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.184285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.193853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b54f-04c8-4539-9340-233eada9520e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.198454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.199099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.199445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.199754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.200482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.202133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.212266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 793/1590] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.213021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.223759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 798/1591] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:25 2022] PUT /identity/v3/projects/647c3ee6b60f4edd92dc08ff7a201d1a/users/927cd6ea4d854f48ad2d92073dd84c55/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:25.238770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b70213-97d2-4a34-8581-318e5fd08a54 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.238949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.283770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.285173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.285485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.285928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] PATH_INFO: `/v3/projects/82d4fd44d8fe4c9b888e417997f4da03/users/7360414f56854b2995b3f78eef666430/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.288531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf0c63-b967-4248-bc42-710574eb5cc4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.293344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.294261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.294545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.294823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] PATH_INFO: `/v3/projects/a02e2e12ebd14e5d8e960a65f41bfb00/users/7bffd469507b4dbc8d3146f051c8c8ad/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.299367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:25.300475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82d4fd44d8fe4c9b888e417997f4da03, user_id=7360414f56854b2995b3f78eef666430, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.302232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:13:25.302232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:13:25.302232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:13:25.304176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.311433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02e2e12ebd14e5d8e960a65f41bfb00, user_id=7bffd469507b4dbc8d3146f051c8c8ad, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.314190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.431932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 794/1592] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:25 2022] PUT /identity/v3/projects/82d4fd44d8fe4c9b888e417997f4da03/users/7360414f56854b2995b3f78eef666430/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:25.432760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.447050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 799/1593] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:25 2022] PUT /identity/v3/projects/a02e2e12ebd14e5d8e960a65f41bfb00/users/7bffd469507b4dbc8d3146f051c8c8ad/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:25.458977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.510570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6854d-ea5a-4953-8611-1653ff261d68 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.513828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08448d91-c81e-4634-b59a-a947767291c5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.514918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.515600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.515884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.516162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.517017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.518402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.518871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.519097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.519446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.519727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.520455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.522077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.529175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 795/1594] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.529853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.532319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 800/1595] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.534035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.543220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fc03ec-9fd3-41bd-88b5-24ed9a967920 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.544318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e51000-372b-4f5b-8674-cb4efd7b662b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.548415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.548876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a51790-8f07-4fd8-9e3f-f5b007a5d3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.549068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.549377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.549564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a51790-8f07-4fd8-9e3f-f5b007a5d3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.549660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.549850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a51790-8f07-4fd8-9e3f-f5b007a5d3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.550131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a51790-8f07-4fd8-9e3f-f5b007a5d3a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.550891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.551427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a51790-8f07-4fd8-9e3f-f5b007a5d3a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.552755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.553094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a51790-8f07-4fd8-9e3f-f5b007a5d3a7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.563719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 796/1597] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.563957 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 801/1597] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.564492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.565825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b650c2-ab97-415b-a254-897a22aa1313 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.567049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b650c2-ab97-415b-a254-897a22aa1313 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.567750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b650c2-ab97-415b-a254-897a22aa1313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.575409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395cc65-5458-4863-bdd6-09507bd9e717 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.580073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.580942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.581225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.581560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] PATH_INFO: `/v3/projects/eb6faba45a60477e8202718b5cb2ca25/users/37e37d4592c44546a502244fbc70cba7/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.586342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f5b650c2-ab97-415b-a254-897a22aa1313 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:25.587559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f5b650c2-ab97-415b-a254-897a22aa1313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:25.595266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 797/1598] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:25.596429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300d9d-9a37-4f0e-b1d4-75ff7ee5ab92 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.596533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb6faba45a60477e8202718b5cb2ca25, user_id=37e37d4592c44546a502244fbc70cba7, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.596720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300d9d-9a37-4f0e-b1d4-75ff7ee5ab92 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.596993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300d9d-9a37-4f0e-b1d4-75ff7ee5ab92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.599345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.615234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1b300d9d-9a37-4f0e-b1d4-75ff7ee5ab92 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:25.616418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1b300d9d-9a37-4f0e-b1d4-75ff7ee5ab92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:25.624208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 798/1599] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:25.625473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3faff03-3b94-4546-9cd1-666b1671f4cf None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.625724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3faff03-3b94-4546-9cd1-666b1671f4cf None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.626004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3faff03-3b94-4546-9cd1-666b1671f4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.639485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:25.654649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a3faff03-3b94-4546-9cd1-666b1671f4cf None None] MFA Rules not processed for user `7360414f56854b2995b3f78eef666430`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:25.655990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a3faff03-3b94-4546-9cd1-666b1671f4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:25.702621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 802/1600] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:25 2022] PUT /identity/v3/projects/eb6faba45a60477e8202718b5cb2ca25/users/37e37d4592c44546a502244fbc70cba7/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:25.705928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.721979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 799/1601] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:25.721979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3faff03-3b94-4546-9cd1-666b1671f4cf None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.733122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3faff03-3b94-4546-9cd1-666b1671f4cf None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.769808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88636afd-8005-4a32-94c2-ede89622bb65 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.774639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.775309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.775591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.775875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.776629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.778264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.792385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 803/1602] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:25.792385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.792655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.792655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.792655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.792959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] PATH_INFO: `/v3/projects/4784aebbfd1d43f896df6ca9057b3251/users/13f9b782a79f48678f297d1ef388da75/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.800640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef6c4a-40ce-4795-aec9-0486f57e8aaa admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:25.805192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:25.806091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:25.806372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:25.806645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] PATH_INFO: `/v3/projects/af776ddc4c0f4e6390301a02c6bbfca0/users/baac9b5e923747e6986720e3ac849056/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:25.808396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4784aebbfd1d43f896df6ca9057b3251, user_id=13f9b782a79f48678f297d1ef388da75, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.811086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.822230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=af776ddc4c0f4e6390301a02c6bbfca0, user_id=baac9b5e923747e6986720e3ac849056, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:25.824859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:25.943359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 800/1603] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:25 2022] PUT /identity/v3/projects/4784aebbfd1d43f896df6ca9057b3251/users/13f9b782a79f48678f297d1ef388da75/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:25.944283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.955094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 804/1604] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:25 2022] PUT /identity/v3/projects/af776ddc4c0f4e6390301a02c6bbfca0/users/baac9b5e923747e6986720e3ac849056/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:25.956208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:25.976914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.006615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc78b732-ab7c-4446-980d-85e604c17f86 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:26.014317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177187a8-97cd-4742-884b-b2181cc228bd admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:26.018062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:26.018710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.018999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.020065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.032581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.067008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:26.068022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.068314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.068594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] PATH_INFO: `/v3/projects/2cac460d93d1472194ce4c01ca7fb7ca/users/0716b07086384fa1bf9e8ca107603436/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.080083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:26.081751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:26.084445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cac460d93d1472194ce4c01ca7fb7ca, user_id=0716b07086384fa1bf9e8ca107603436, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:26.087321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:26.091804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 805/1605] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:26.092611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:26.102450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76deab15-92bf-4bf8-8999-1a84148a5964 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:26.105176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ca5ed-9c66-403f-ae77-dc1959e1c7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:26.105835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ca5ed-9c66-403f-ae77-dc1959e1c7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.106114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ca5ed-9c66-403f-ae77-dc1959e1c7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.106438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ca5ed-9c66-403f-ae77-dc1959e1c7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.116244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ca5ed-9c66-403f-ae77-dc1959e1c7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:26.117697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ca5ed-9c66-403f-ae77-dc1959e1c7b1 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:26.127241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 806/1606] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:26.131269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a3c64-4083-4ab5-ba20-021cf2a5eabf None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.131269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a3c64-4083-4ab5-ba20-021cf2a5eabf None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.131269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a3c64-4083-4ab5-ba20-021cf2a5eabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.158108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 801/1607] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:25 2022] PUT /identity/v3/projects/2cac460d93d1472194ce4c01ca7fb7ca/users/0716b07086384fa1bf9e8ca107603436/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:26.163054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:26.169203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ad5a3c64-4083-4ab5-ba20-021cf2a5eabf None None] MFA Rules not processed for user `37e37d4592c44546a502244fbc70cba7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:26.170539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ad5a3c64-4083-4ab5-ba20-021cf2a5eabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:26.244265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3816f29-5072-4374-96ca-546c51a18be1 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:26.251374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ffcc-0123-46a9-af88-2f8cb4c2877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:26.251374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ffcc-0123-46a9-af88-2f8cb4c2877c admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.251374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ffcc-0123-46a9-af88-2f8cb4c2877c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.251374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ffcc-0123-46a9-af88-2f8cb4c2877c admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.251774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ffcc-0123-46a9-af88-2f8cb4c2877c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:26.254609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ffcc-0123-46a9-af88-2f8cb4c2877c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:26.260421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 807/1608] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:26.261493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a3c64-4083-4ab5-ba20-021cf2a5eabf None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:26.273599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a3c64-4083-4ab5-ba20-021cf2a5eabf None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:26.282609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a540efa-0958-4f60-b761-5208b4a47e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:26.283744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a540efa-0958-4f60-b761-5208b4a47e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.284075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a540efa-0958-4f60-b761-5208b4a47e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.284358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a540efa-0958-4f60-b761-5208b4a47e71 admin admin] PATH_INFO: `/v3/projects/35a9ad5293a844f4a481f73dcee4f86b/users/e59aab502a9a43189637822d3dd2be57/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.302986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a540efa-0958-4f60-b761-5208b4a47e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a9ad5293a844f4a481f73dcee4f86b, user_id=e59aab502a9a43189637822d3dd2be57, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:26.306193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a540efa-0958-4f60-b761-5208b4a47e71 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:26.411608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 802/1609] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/projects => generated 381 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:26.414812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2320efb-b8b8-43d0-8510-51589f7bc338 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.415119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2320efb-b8b8-43d0-8510-51589f7bc338 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.415401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2320efb-b8b8-43d0-8510-51589f7bc338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.433527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a2320efb-b8b8-43d0-8510-51589f7bc338 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:26.434851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a2320efb-b8b8-43d0-8510-51589f7bc338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:26.480472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 808/1610] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:26 2022] PUT /identity/v3/projects/35a9ad5293a844f4a481f73dcee4f86b/users/e59aab502a9a43189637822d3dd2be57/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:26.482070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6737985-987a-4548-83fd-a73ea308e700 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.482361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6737985-987a-4548-83fd-a73ea308e700 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.483059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6737985-987a-4548-83fd-a73ea308e700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.506303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 803/1611] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:26.506933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f6737985-987a-4548-83fd-a73ea308e700 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:26.507818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf9aa2-b83c-473d-b65a-4570c38f7bdc None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.508110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf9aa2-b83c-473d-b65a-4570c38f7bdc None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.508291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f6737985-987a-4548-83fd-a73ea308e700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:26.508423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf9aa2-b83c-473d-b65a-4570c38f7bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.536502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.546798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-fdbf9aa2-b83c-473d-b65a-4570c38f7bdc None None] MFA Rules not processed for user `0716b07086384fa1bf9e8ca107603436`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:26.548474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fdbf9aa2-b83c-473d-b65a-4570c38f7bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:26.583069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:26.617978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 809/1612] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:26.619065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6737985-987a-4548-83fd-a73ea308e700 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:26.632515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6737985-987a-4548-83fd-a73ea308e700 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:26.638369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e66576-f3b8-4db3-a593-69de7d5e5e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:26.639200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e66576-f3b8-4db3-a593-69de7d5e5e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.639607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e66576-f3b8-4db3-a593-69de7d5e5e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.639989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e66576-f3b8-4db3-a593-69de7d5e5e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.640971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e66576-f3b8-4db3-a593-69de7d5e5e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:26.642956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e66576-f3b8-4db3-a593-69de7d5e5e36 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:26.668142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 804/1613] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:26.671459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf9aa2-b83c-473d-b65a-4570c38f7bdc None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:26.683753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf9aa2-b83c-473d-b65a-4570c38f7bdc None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:26.688852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:26.689572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.689859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.690139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.691563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:26.693763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:26.797520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 810/1614] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/projects => generated 401 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:26.799242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:26.803356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:26.803356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:26.835506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] MFA Rules not processed for user `e59aab502a9a43189637822d3dd2be57`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:26.837479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:26.866498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 805/1615] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:26.866498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:26.866498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:26.924226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 811/1616] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:26.926073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:26.927857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:26.957595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7440a7e-4906-4b73-a21a-265c6d6c8fdd admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.012942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b22a5b-30ef-457d-87d3-8e36dd150b88 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.018752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c45d32-ffa9-47e7-932f-69f45097049b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.019459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c45d32-ffa9-47e7-932f-69f45097049b service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.019561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c45d32-ffa9-47e7-932f-69f45097049b service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.019782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c45d32-ffa9-47e7-932f-69f45097049b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.065333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.066576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.067049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.067330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.084169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:27.106949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c45d32-ffa9-47e7-932f-69f45097049b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.108633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c45d32-ffa9-47e7-932f-69f45097049b service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.127076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 812/1617] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:27.130135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ca15d-d4d4-4a8e-b686-a02ae6792740 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.130599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ca15d-d4d4-4a8e-b686-a02ae6792740 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.131644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ca15d-d4d4-4a8e-b686-a02ae6792740 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.135028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 813/1618] 10.222.0.35 () {54 vars in 961 bytes} [Tue May 24 10:13:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:27.136589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ca15d-d4d4-4a8e-b686-a02ae6792740 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.167479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ca15d-d4d4-4a8e-b686-a02ae6792740 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.168179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.169883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.174913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.176362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.177042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.177347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.178057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.180948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.182745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 806/1619] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:27.183658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.194282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f90ab-ed02-44b0-808a-0459485098f3 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.199566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d5b32-9cb5-4155-819a-11a79cdcb9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.200512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5b32-9cb5-4155-819a-11a79cdcb9bf admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.201091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5b32-9cb5-4155-819a-11a79cdcb9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.201478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d5b32-9cb5-4155-819a-11a79cdcb9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.208062 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d5b32-9cb5-4155-819a-11a79cdcb9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.208062 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d5b32-9cb5-4155-819a-11a79cdcb9bf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.215681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 807/1620] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:27.217314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9ec0c-3975-4638-87f1-a559b75b3566 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.217833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9ec0c-3975-4638-87f1-a559b75b3566 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.218402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9ec0c-3975-4638-87f1-a559b75b3566 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.220050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 808/1621] 10.222.0.35 () {54 vars in 961 bytes} [Tue May 24 10:13:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:27.223549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ee210-739a-4255-bcb8-c2bd8128bff1 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.223905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ee210-739a-4255-bcb8-c2bd8128bff1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.223905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ee210-739a-4255-bcb8-c2bd8128bff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.231259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:27.244225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a95ee210-739a-4255-bcb8-c2bd8128bff1 None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:27.245750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a95ee210-739a-4255-bcb8-c2bd8128bff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:27.276747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 814/1622] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:27 2022] POST /identity/v3/users => generated 307 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:27.278187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.290204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770d024-506e-4f4b-be60-4d37dadb18a4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.297902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.298808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.299094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.299406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] PATH_INFO: `/v3/projects/7cc5b120f5b6434b89a7686298bea31c/users/8a8e9ccbef474093861aaf61ed5ac37d/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.322778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc5b120f5b6434b89a7686298bea31c, user_id=8a8e9ccbef474093861aaf61ed5ac37d, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.325702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.331547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 809/1623] 10.222.0.35 () {58 vars in 1058 bytes} [Tue May 24 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:27.333532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30100c12-92b3-4222-940a-090b6dfffd8b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.334006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30100c12-92b3-4222-940a-090b6dfffd8b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.338822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30100c12-92b3-4222-940a-090b6dfffd8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.354278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-30100c12-92b3-4222-940a-090b6dfffd8b None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:27.355759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-30100c12-92b3-4222-940a-090b6dfffd8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:27.366041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 810/1624] 10.222.0.35 () {58 vars in 1058 bytes} [Tue May 24 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:27.367475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30100c12-92b3-4222-940a-090b6dfffd8b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.377099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30100c12-92b3-4222-940a-090b6dfffd8b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.381850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.382641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.383028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.383374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.384172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.386058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.400354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 811/1625] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:27.400354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.414324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa8619-3f83-4a3d-ac59-529f5b31b529 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.419836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.420487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.420770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.421074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.439039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 815/1626] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:27 2022] PUT /identity/v3/projects/7cc5b120f5b6434b89a7686298bea31c/users/8a8e9ccbef474093861aaf61ed5ac37d/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:27.439039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.485773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.487570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.498730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 812/1627] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:27.499693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.511033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a1af5-34e5-4366-9a28-533fb0f16ff1 service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.546741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70496af-8c7d-463a-9a38-177eda31cd6c admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.559298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.560266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.560732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.561376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.582753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.583098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.584186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.584564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] PATH_INFO: `/v3/projects/5de3661c07d34c4fbf2f27dd3b4cc0cf/users/14044d8c7fe0480088c3d06d90f91231/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.607816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de3661c07d34c4fbf2f27dd3b4cc0cf, user_id=14044d8c7fe0480088c3d06d90f91231, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.610600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.648999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.650764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.662607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 816/1628] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:27.663443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.675675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56833dc1-4a59-41db-96fd-b99ba0033608 service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.681363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.682093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.682558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.682845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.683830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.685474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.699477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 817/1629] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:27.703122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.712052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 813/1630] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:27 2022] PUT /identity/v3/projects/5de3661c07d34c4fbf2f27dd3b4cc0cf/users/14044d8c7fe0480088c3d06d90f91231/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:27.712052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.718705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e21c59-5023-4af6-8812-2eb7ecf8f972 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.795264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea19e8c-e013-4a62-83c0-9339fc8c8a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.796257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea19e8c-e013-4a62-83c0-9339fc8c8a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.796627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea19e8c-e013-4a62-83c0-9339fc8c8a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.796943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea19e8c-e013-4a62-83c0-9339fc8c8a88 admin admin] PATH_INFO: `/v3/projects/7cc5b120f5b6434b89a7686298bea31c/users/8a8e9ccbef474093861aaf61ed5ac37d/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.799350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea19e8c-e013-4a62-83c0-9339fc8c8a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc5b120f5b6434b89a7686298bea31c, user_id=8a8e9ccbef474093861aaf61ed5ac37d, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.803592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea19e8c-e013-4a62-83c0-9339fc8c8a88 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.819867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 818/1631] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:27 2022] PUT /identity/v3/projects/7cc5b120f5b6434b89a7686298bea31c/users/8a8e9ccbef474093861aaf61ed5ac37d/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:13:27.822845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90daada-60f6-4f52-819b-08b414e20599 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.822845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90daada-60f6-4f52-819b-08b414e20599 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.822845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90daada-60f6-4f52-819b-08b414e20599 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.823963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 819/1632] 10.222.0.35 () {54 vars in 912 bytes} [Tue May 24 10:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:27.827142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2221be83-dee1-4b58-9e02-daafb9af0c3d admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.830789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de60ed2-926e-4278-b111-b6984fdcfe58 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.831139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de60ed2-926e-4278-b111-b6984fdcfe58 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.831468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de60ed2-926e-4278-b111-b6984fdcfe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.858383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6de60ed2-926e-4278-b111-b6984fdcfe58 None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:27.859596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6de60ed2-926e-4278-b111-b6984fdcfe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:27.956100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494bf35-a64c-4313-adc4-de3ff0c06b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.956100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494bf35-a64c-4313-adc4-de3ff0c06b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.956100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494bf35-a64c-4313-adc4-de3ff0c06b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.956100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494bf35-a64c-4313-adc4-de3ff0c06b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.957213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494bf35-a64c-4313-adc4-de3ff0c06b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:27.957213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494bf35-a64c-4313-adc4-de3ff0c06b58 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:27.967432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 814/1633] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:13:27.967432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292160-bc52-4e77-9608-d094026d9099 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.967803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292160-bc52-4e77-9608-d094026d9099 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.968035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f292160-bc52-4e77-9608-d094026d9099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.970505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 820/1634] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:27.971557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de60ed2-926e-4278-b111-b6984fdcfe58 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:27.985050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de60ed2-926e-4278-b111-b6984fdcfe58 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:27.992730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:27.994219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:27.994897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:27.995567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] PATH_INFO: `/v3/projects/5de3661c07d34c4fbf2f27dd3b4cc0cf/users/14044d8c7fe0480088c3d06d90f91231/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:27.998154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de3661c07d34c4fbf2f27dd3b4cc0cf, user_id=14044d8c7fe0480088c3d06d90f91231, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:28.001946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:28.016995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 821/1635] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:27 2022] PUT /identity/v3/projects/5de3661c07d34c4fbf2f27dd3b4cc0cf/users/14044d8c7fe0480088c3d06d90f91231/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:28.017991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:28.021585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1f292160-bc52-4e77-9608-d094026d9099 None None] MFA Rules not processed for user `8a8e9ccbef474093861aaf61ed5ac37d`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:28.040482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1f292160-bc52-4e77-9608-d094026d9099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:28.104812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18ec0d-0141-44cf-b42b-dfc77c2b91f1 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:28.109925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84249d-46ff-4dda-8782-a11e363a26f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:28.110679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84249d-46ff-4dda-8782-a11e363a26f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.110904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84249d-46ff-4dda-8782-a11e363a26f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.111209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84249d-46ff-4dda-8782-a11e363a26f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.118236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 815/1636] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:28.118236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c2e2-610f-4b7b-a3f6-4d331f647c67 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.118236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c2e2-610f-4b7b-a3f6-4d331f647c67 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.118236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c2e2-610f-4b7b-a3f6-4d331f647c67 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.118236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 816/1637] 10.222.0.35 () {54 vars in 912 bytes} [Tue May 24 10:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:28.128706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a90a6-0345-4c32-aca5-dd796109b1ed None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.128706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a90a6-0345-4c32-aca5-dd796109b1ed None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.128706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a90a6-0345-4c32-aca5-dd796109b1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.132218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:28.153986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5b8a90a6-0345-4c32-aca5-dd796109b1ed None None] MFA Rules not processed for user `14044d8c7fe0480088c3d06d90f91231`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:28.157589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5b8a90a6-0345-4c32-aca5-dd796109b1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:28.207956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84249d-46ff-4dda-8782-a11e363a26f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:28.209510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84249d-46ff-4dda-8782-a11e363a26f7 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:28.212320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:28.225389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 822/1638] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:28.227695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364e420-5394-4710-b4da-55a795500d5d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.227994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364e420-5394-4710-b4da-55a795500d5d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.228270 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364e420-5394-4710-b4da-55a795500d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.247121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0364e420-5394-4710-b4da-55a795500d5d None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:28.248364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0364e420-5394-4710-b4da-55a795500d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:28.259518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 823/1639] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:28.265977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364e420-5394-4710-b4da-55a795500d5d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:28.266130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 817/1640] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:28.270382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364e420-5394-4710-b4da-55a795500d5d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:28.273914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:28.274634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.274915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.275246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.288011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:28.289449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:28.299937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 824/1641] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:28.384840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed0c54-33d9-4322-a38a-3ee953d25125 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.385143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed0c54-33d9-4322-a38a-3ee953d25125 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.385577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed0c54-33d9-4322-a38a-3ee953d25125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.426693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-25ed0c54-33d9-4322-a38a-3ee953d25125 None None] MFA Rules not processed for user `927cd6ea4d854f48ad2d92073dd84c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:28.428704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-25ed0c54-33d9-4322-a38a-3ee953d25125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:28.511348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:28.536758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c24a9-5433-47e3-9a81-61620277fea3 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:28.545519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:28.546472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.546980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.551605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.570017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 818/1642] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:28.575010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed0c54-33d9-4322-a38a-3ee953d25125 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:28.583215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:28.586541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:28.610505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 825/1643] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:28.661362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed0c54-33d9-4322-a38a-3ee953d25125 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:28.666748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:28.667694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.668900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.669282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.670099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:28.672478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:28.776592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 819/1644] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:28 2022] POST /identity/v3/projects => generated 401 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:28.778243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:28.793601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3fea7e-1972-4529-8f32-08772a02a791 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:28.869693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:28.870961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:28.871413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:28.871776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:28.872630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:28.874990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:28.932955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:29.019975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 826/1645] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:28 2022] POST /identity/v3/users => generated 307 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:29.021332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:29.032648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74993f23-19c9-4c75-b863-b00543fece00 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:29.039277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:29.039966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:29.040254 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:29.040534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:29.042385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:29.043388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:29.061030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 820/1646] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:29.064751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:29.079477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d7027-2eb4-4e8f-bbac-845f71666c2b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:29.092523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:29.095467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:29.095467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:29.095467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] PATH_INFO: `/v3/projects/b075bb5c53004a02a527e4e3ac0cddc4/users/7bba46cfdba14b6daebfa9c5ae534d19/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:29.121970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b075bb5c53004a02a527e4e3ac0cddc4, user_id=7bba46cfdba14b6daebfa9c5ae534d19, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:29.124752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:29.190441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:29.233948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:29.234672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 827/1647] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:29 2022] PUT /identity/v3/projects/b075bb5c53004a02a527e4e3ac0cddc4/users/7bba46cfdba14b6daebfa9c5ae534d19/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:29.248232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058588e-4518-4e04-a35c-b8721f0e33a5 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:29.318548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:29.319449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:29.319734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:29.320073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:29.320865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:29.322732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:29.345049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 821/1648] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:13:29.349612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:29.362601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd06598d-8f50-4ffa-bbd1-e2559aa93782 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:29.369341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b0b71-ae96-4b98-8973-28535e5e6901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:29.370756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0b71-ae96-4b98-8973-28535e5e6901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:29.371062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0b71-ae96-4b98-8973-28535e5e6901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:29.371347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b0b71-ae96-4b98-8973-28535e5e6901 admin admin] PATH_INFO: `/v3/projects/b075bb5c53004a02a527e4e3ac0cddc4/users/7bba46cfdba14b6daebfa9c5ae534d19/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:29.373534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b0b71-ae96-4b98-8973-28535e5e6901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b075bb5c53004a02a527e4e3ac0cddc4, user_id=7bba46cfdba14b6daebfa9c5ae534d19, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:29.377177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b0b71-ae96-4b98-8973-28535e5e6901 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:29.391973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 828/1649] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:29 2022] PUT /identity/v3/projects/b075bb5c53004a02a527e4e3ac0cddc4/users/7bba46cfdba14b6daebfa9c5ae534d19/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:29.669600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:29.679386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9f918-a7b0-436e-9db0-f01a429dd672 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:29.679684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9f918-a7b0-436e-9db0-f01a429dd672 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:29.679989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9f918-a7b0-436e-9db0-f01a429dd672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:29.728205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-52d9f918-a7b0-436e-9db0-f01a429dd672 None None] MFA Rules not processed for user `13f9b782a79f48678f297d1ef388da75`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:29.730405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-52d9f918-a7b0-436e-9db0-f01a429dd672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:30.015202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:30.015202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba77546-1e6d-4f76-b490-36b990e7daa9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:30.015202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:30.015202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:30.016289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:30.016289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:30.017523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 829/1650] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:30.017523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f71a6-ea71-42b5-b71a-314040bd12d5 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:30.017523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f71a6-ea71-42b5-b71a-314040bd12d5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:30.017523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f71a6-ea71-42b5-b71a-314040bd12d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:30.017523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-448f71a6-ea71-42b5-b71a-314040bd12d5 None None] MFA Rules not processed for user `13f9b782a79f48678f297d1ef388da75`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:30.017523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-448f71a6-ea71-42b5-b71a-314040bd12d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:30.021113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 830/1651] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:30.021627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:30.021627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:30.031968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 822/1652] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:30.223319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f71a6-ea71-42b5-b71a-314040bd12d5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:30.232845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:30.240824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa3ac20-245d-49b7-a3c0-3c3c42eef40f service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:30.252744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:30.267581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:30.267581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:30.267581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:30.371941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f71a6-ea71-42b5-b71a-314040bd12d5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:30.381839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28b6e8-2d5a-4126-b4c1-928e025e6845 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:30.383940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28b6e8-2d5a-4126-b4c1-928e025e6845 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:30.384557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28b6e8-2d5a-4126-b4c1-928e025e6845 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:30.393520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28b6e8-2d5a-4126-b4c1-928e025e6845 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:30.393651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:30.395137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:30.406869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28b6e8-2d5a-4126-b4c1-928e025e6845 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:30.408638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28b6e8-2d5a-4126-b4c1-928e025e6845 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:30.408850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 823/1653] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:13:30.420227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 831/1654] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:30.548805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:30.601249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:30.611761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e144157-3aa8-430f-b991-ec2d67db7e2a service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:30.616720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40535505-80ab-49e3-80f1-83800345cecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:30.617432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40535505-80ab-49e3-80f1-83800345cecb service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:30.617723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40535505-80ab-49e3-80f1-83800345cecb service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:30.618055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40535505-80ab-49e3-80f1-83800345cecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:30.746866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40535505-80ab-49e3-80f1-83800345cecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:30.748579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40535505-80ab-49e3-80f1-83800345cecb service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:30.764076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 824/1655] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:31.432106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415b750-9565-4521-98a1-6d7d8f6a49c2 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:31.432632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415b750-9565-4521-98a1-6d7d8f6a49c2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:31.432701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415b750-9565-4521-98a1-6d7d8f6a49c2 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:31.435248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 832/1656] 10.222.0.35 () {54 vars in 912 bytes} [Tue May 24 10:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:31.444761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312a16e-26d4-4d36-8ea3-817e0a8fb65f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:31.445251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312a16e-26d4-4d36-8ea3-817e0a8fb65f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:31.445381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312a16e-26d4-4d36-8ea3-817e0a8fb65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:31.497935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e312a16e-26d4-4d36-8ea3-817e0a8fb65f None None] MFA Rules not processed for user `9937f1088b304574935fcdf579e4de7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:31.500228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e312a16e-26d4-4d36-8ea3-817e0a8fb65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:31.518376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecb973-ab55-4231-9ee5-45ad6adcc547 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:31.519341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecb973-ab55-4231-9ee5-45ad6adcc547 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:31.519751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcecb973-ab55-4231-9ee5-45ad6adcc547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:31.596204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fcecb973-ab55-4231-9ee5-45ad6adcc547 None None] MFA Rules not processed for user `7bba46cfdba14b6daebfa9c5ae534d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:31.597490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fcecb973-ab55-4231-9ee5-45ad6adcc547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:31.623467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:31.687802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 825/1657] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:13:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:13:31.794165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 833/1658] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:31 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:31.870972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312a16e-26d4-4d36-8ea3-817e0a8fb65f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:31.883035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312a16e-26d4-4d36-8ea3-817e0a8fb65f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:31.891229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b637ce8-dba4-4d0f-a1ea-2aa769572df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:31.891229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b637ce8-dba4-4d0f-a1ea-2aa769572df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:31.891469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b637ce8-dba4-4d0f-a1ea-2aa769572df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:31.892684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b637ce8-dba4-4d0f-a1ea-2aa769572df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:31.906486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b637ce8-dba4-4d0f-a1ea-2aa769572df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:31.908122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b637ce8-dba4-4d0f-a1ea-2aa769572df7 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:31.919336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 826/1659] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:33.199623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55338fff-56bb-446a-97e8-b2be5e864002 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:33.200950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55338fff-56bb-446a-97e8-b2be5e864002 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:33.201406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55338fff-56bb-446a-97e8-b2be5e864002 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:33.203928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 834/1660] 10.222.0.35 () {54 vars in 960 bytes} [Tue May 24 10:13:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:33.211983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f180c-10de-4bbc-80ff-8eea1f26fcdd None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:33.212560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f180c-10de-4bbc-80ff-8eea1f26fcdd None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:33.212831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f180c-10de-4bbc-80ff-8eea1f26fcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:33.236749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c55f180c-10de-4bbc-80ff-8eea1f26fcdd None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:33.239361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c55f180c-10de-4bbc-80ff-8eea1f26fcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:33.303766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724875b-1d7f-4ccf-af2d-22998e79bebe None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:33.304053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724875b-1d7f-4ccf-af2d-22998e79bebe None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:33.304494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724875b-1d7f-4ccf-af2d-22998e79bebe None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:33.311326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 835/1661] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:33.313439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094136b3-2a19-4b6a-b1d0-547fa8e82dad None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:33.313730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094136b3-2a19-4b6a-b1d0-547fa8e82dad None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:33.314097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094136b3-2a19-4b6a-b1d0-547fa8e82dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:33.316469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:33.357615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-094136b3-2a19-4b6a-b1d0-547fa8e82dad None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:33.359040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-094136b3-2a19-4b6a-b1d0-547fa8e82dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:33.391720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 827/1662] 10.222.0.35 () {58 vars in 1057 bytes} [Tue May 24 10:13:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:33.412496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f180c-10de-4bbc-80ff-8eea1f26fcdd None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:33.440496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:33.457035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f180c-10de-4bbc-80ff-8eea1f26fcdd None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:33.480309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 836/1663] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:13:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:13:33.511362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:33.512507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:33.514986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:33.515208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:33.528418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:33.531675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094136b3-2a19-4b6a-b1d0-547fa8e82dad None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:33.548465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094136b3-2a19-4b6a-b1d0-547fa8e82dad None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:33.564207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5d147-d453-4df5-86f8-99c9d05470d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:33.569498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5d147-d453-4df5-86f8-99c9d05470d0 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:33.569498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5d147-d453-4df5-86f8-99c9d05470d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:33.569498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5d147-d453-4df5-86f8-99c9d05470d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:33.573956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:33.579079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:33.595139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5d147-d453-4df5-86f8-99c9d05470d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:33.599929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5d147-d453-4df5-86f8-99c9d05470d0 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:33.609310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 828/1664] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:33.647268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 837/1665] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:13:33.686832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:33.926428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e231e93-c48f-4847-bf7f-2ed43631ddb9 service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:33.944241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:33.945173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:33.945506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:33.945787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] PATH_INFO: `/v3/users/0716b07086384fa1bf9e8ca107603436` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:33.948807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0716b07086384fa1bf9e8ca107603436)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:33.951035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: WARNING py.warnings [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:13:33.951035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: warnings.warn(msg) May 24 10:13:33.951035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: May 24 10:13:33.958686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:34.373356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0716b07086384fa1bf9e8ca107603436'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:34.374074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0716b07086384fa1bf9e8ca107603436'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:34.411310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0716b07086384fa1bf9e8ca107603436'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:34.413398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0716b07086384fa1bf9e8ca107603436'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:34.450638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0716b07086384fa1bf9e8ca107603436'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:34.535848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 829/1666] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:33 2022] DELETE /identity/v3/users/0716b07086384fa1bf9e8ca107603436 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:13:34.542916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:34.543503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:34.543503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:34.572382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:34.573947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:34.659587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 838/1667] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:34.687252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:34.688578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:34.802657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:34.834926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41182063-591c-4438-9c81-69a71716f4ac admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:34.839400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:34.840152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:34.841530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:34.841892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:34.867079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:34.867079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:34.895618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 830/1668] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:36.058720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:36.060751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:36.086343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2525b-8af1-4cdb-8238-67c2178c07e3 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:36.107507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:36.108952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:36.109628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:36.110111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] PATH_INFO: `/v3/users/927cd6ea4d854f48ad2d92073dd84c55` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:36.119273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] RBAC: Authorizing `identity:delete_user(user_id=927cd6ea4d854f48ad2d92073dd84c55)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:36.119273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:36.119437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:36.120105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:36.146522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2259e41-8e0e-4f67-be59-3d8296aa55c2 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:36.165659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:36.168098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:36.168743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:36.169207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] PATH_INFO: `/v3/projects/2cac460d93d1472194ce4c01ca7fb7ca` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:36.174298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cac460d93d1472194ce4c01ca7fb7ca)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:36.185863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:36.513551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cac460d93d1472194ce4c01ca7fb7ca', 'request_id': 'req-74c96261-4701-44cd-b8ce-9942b1f18031'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.513752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927cd6ea4d854f48ad2d92073dd84c55'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.533108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927cd6ea4d854f48ad2d92073dd84c55'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.542263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927cd6ea4d854f48ad2d92073dd84c55'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.570900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927cd6ea4d854f48ad2d92073dd84c55'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.668466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cac460d93d1472194ce4c01ca7fb7ca', 'request_id': 'req-74c96261-4701-44cd-b8ce-9942b1f18031'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.676038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cac460d93d1472194ce4c01ca7fb7ca', 'request_id': 'req-74c96261-4701-44cd-b8ce-9942b1f18031'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.708048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] The token cache is being invalidate because project 2cac460d93d1472194ce4c01ca7fb7ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:36.708532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.709529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-74c96261-4701-44cd-b8ce-9942b1f18031 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.711608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 831/1669] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:36 2022] DELETE /identity/v3/projects/2cac460d93d1472194ce4c01ca7fb7ca => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:36.716355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e8f7-7d75-4eca-807f-de7a2cb880c0 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:36.716815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e8f7-7d75-4eca-807f-de7a2cb880c0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:36.717099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd71e8f7-7d75-4eca-807f-de7a2cb880c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:36.788158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7a0ac00a-04df-4f2d-b78f-4665ba75285b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927cd6ea4d854f48ad2d92073dd84c55'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:36.798639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 839/1670] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:36 2022] DELETE /identity/v3/users/927cd6ea4d854f48ad2d92073dd84c55 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:36.964004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-bd71e8f7-7d75-4eca-807f-de7a2cb880c0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:36.966389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bd71e8f7-7d75-4eca-807f-de7a2cb880c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:37.073870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 832/1671] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:37.082754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04624-26db-487a-8682-d3174bca1375 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:37.083241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04624-26db-487a-8682-d3174bca1375 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:37.083398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa04624-26db-487a-8682-d3174bca1375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:37.117159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1fa04624-26db-487a-8682-d3174bca1375 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:37.118739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1fa04624-26db-487a-8682-d3174bca1375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:37.138319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 840/1672] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:37.141498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71e8f7-7d75-4eca-807f-de7a2cb880c0 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:37.164875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd71e8f7-7d75-4eca-807f-de7a2cb880c0 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:37.181920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:37.189224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:37.189224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:37.189224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:37.189224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:37.189224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:37.191967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:37.207960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 833/1673] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:37.217528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa04624-26db-487a-8682-d3174bca1375 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:37.236490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa04624-26db-487a-8682-d3174bca1375 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:37.243336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:37.244390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:37.247931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:37.247931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:37.247931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:37.249182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:37.431240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 841/1674] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:37 2022] POST /identity/v3/projects => generated 387 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:37.437590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:37.448113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326caee3-af73-40dd-9621-23c24bea87b3 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:37.546178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:37.548947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:37.561295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:37.570055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:37.570508 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:37.571031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:37.572458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:37.592688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:37.593893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:37.691903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0855c-7724-464f-953a-6a57bf2ef0c5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:37.700896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:37.703377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:37.703825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:37.704688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] PATH_INFO: `/v3/projects/647c3ee6b60f4edd92dc08ff7a201d1a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:37.709618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] RBAC: Authorizing `identity:delete_project(project_id=647c3ee6b60f4edd92dc08ff7a201d1a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:37.712128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: WARNING py.warnings [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 24 10:13:37.712128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: warnings.warn(msg) May 24 10:13:37.712128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: May 24 10:13:37.715044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:37.763868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 834/1675] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:37 2022] POST /identity/v3/users => generated 294 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:37.767897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:37.783456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159c476-99b3-497a-99dc-6b6551aac888 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:37.789092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:37.790138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:37.790568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:37.790852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:37.791761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:37.798263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:37.811702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 835/1676] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:37.813205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:37.851836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008cb900-70ef-49bc-94b6-9e71c2a65e3b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:37.858896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:37.860065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:37.860375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:37.860657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:37.861444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:37.864128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:37.884102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 836/1677] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:37.903003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:37.935556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ce1ae8-df5f-4ad2-a0c3-584d5e999982 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:37.981748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647c3ee6b60f4edd92dc08ff7a201d1a', 'request_id': 'req-49884cfd-e0de-4669-9137-1fa1fc1d4314'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.085713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647c3ee6b60f4edd92dc08ff7a201d1a', 'request_id': 'req-49884cfd-e0de-4669-9137-1fa1fc1d4314'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.096484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b94fc2-525d-4f33-895a-0b1c46703349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:38.097966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94fc2-525d-4f33-895a-0b1c46703349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:38.098268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94fc2-525d-4f33-895a-0b1c46703349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:38.098509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b94fc2-525d-4f33-895a-0b1c46703349 admin admin] PATH_INFO: `/v3/projects/4ffb58b3629f4ed3953f34bf901624c7/users/335daaee431c47bc9f3274b4475f8963/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:38.113677 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647c3ee6b60f4edd92dc08ff7a201d1a', 'request_id': 'req-49884cfd-e0de-4669-9137-1fa1fc1d4314'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.134791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b94fc2-525d-4f33-895a-0b1c46703349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ffb58b3629f4ed3953f34bf901624c7, user_id=335daaee431c47bc9f3274b4475f8963, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:38.137945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b94fc2-525d-4f33-895a-0b1c46703349 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:38.158297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] The token cache is being invalidate because project 647c3ee6b60f4edd92dc08ff7a201d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:38.158603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.160596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.164840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 842/1678] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:37 2022] DELETE /identity/v3/projects/647c3ee6b60f4edd92dc08ff7a201d1a => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:38.169219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:38.171294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:38.212765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49884cfd-e0de-4669-9137-1fa1fc1d4314 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:38.228701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:38.229650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:38.230098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:38.230548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] PATH_INFO: `/v3/users/7bba46cfdba14b6daebfa9c5ae534d19` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:38.232926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bba46cfdba14b6daebfa9c5ae534d19)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:38.236660 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:38.289031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 837/1679] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:37 2022] PUT /identity/v3/projects/4ffb58b3629f4ed3953f34bf901624c7/users/335daaee431c47bc9f3274b4475f8963/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:38.290348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:38.290674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:38.290960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:38.400280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] MFA Rules not processed for user `335daaee431c47bc9f3274b4475f8963`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:38.401930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:38.535499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:38.601738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 838/1680] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:13:38.602654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:38.606262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:38.699108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bba46cfdba14b6daebfa9c5ae534d19'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.730755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bba46cfdba14b6daebfa9c5ae534d19'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.734459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bba46cfdba14b6daebfa9c5ae534d19'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.782214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bba46cfdba14b6daebfa9c5ae534d19'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.835500 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a254c-d311-41cc-a492-2fa6e058509d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:38.843580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:38.844782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:38.845190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:38.845629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] PATH_INFO: `/v3/users/37e37d4592c44546a502244fbc70cba7` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:38.855514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e37d4592c44546a502244fbc70cba7)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:38.862876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bba46cfdba14b6daebfa9c5ae534d19'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:38.865108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:38.873032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 843/1681] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:38 2022] DELETE /identity/v3/users/7bba46cfdba14b6daebfa9c5ae534d19 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:38.873032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:38.874376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:39.195586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c5a74-05ef-4516-a08c-8c1ba4c49009 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:39.202201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8105ec-000e-47e4-881a-2a19b0af35cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:39.203251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8105ec-000e-47e4-881a-2a19b0af35cd admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:39.203959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8105ec-000e-47e4-881a-2a19b0af35cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:39.204655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8105ec-000e-47e4-881a-2a19b0af35cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:39.206155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8105ec-000e-47e4-881a-2a19b0af35cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:39.208273 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e37d4592c44546a502244fbc70cba7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:39.209183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e37d4592c44546a502244fbc70cba7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:39.213627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8105ec-000e-47e4-881a-2a19b0af35cd admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:39.223425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e37d4592c44546a502244fbc70cba7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:39.225855 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e37d4592c44546a502244fbc70cba7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:39.250629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e37d4592c44546a502244fbc70cba7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:39.361308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 844/1682] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:38 2022] POST /identity/v3/projects => generated 385 bytes in 490 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:39.369241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610af9d-9a8d-485b-bd9e-f319839f4457 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:39.373313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610af9d-9a8d-485b-bd9e-f319839f4457 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:39.375022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610af9d-9a8d-485b-bd9e-f319839f4457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:39.386725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 839/1683] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:38 2022] DELETE /identity/v3/users/37e37d4592c44546a502244fbc70cba7 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:39.391075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:39.392892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:39.456438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a610af9d-9a8d-485b-bd9e-f319839f4457 None None] MFA Rules not processed for user `7bffd469507b4dbc8d3146f051c8c8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:39.459884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a610af9d-9a8d-485b-bd9e-f319839f4457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:39.517498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05981b3-887a-43ac-b2ac-9950a0bb5e34 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:39.524266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:39.525098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:39.525486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:39.525925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:39.526731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:39.532970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:39.641222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 845/1684] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:39.642793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2522ff-f97c-4f21-94d0-b7c8e01dc172 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:39.643173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2522ff-f97c-4f21-94d0-b7c8e01dc172 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:39.648180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2522ff-f97c-4f21-94d0-b7c8e01dc172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:39.669695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8f2522ff-f97c-4f21-94d0-b7c8e01dc172 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:39.671234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8f2522ff-f97c-4f21-94d0-b7c8e01dc172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:39.686377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 846/1685] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:39.688008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:39.688443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:39.688851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:39.725532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] MFA Rules not processed for user `baac9b5e923747e6986720e3ac849056`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:39.727096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:39.794110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 840/1686] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:39 2022] POST /identity/v3/users => generated 299 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:39.795052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:39.808225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:39.885305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 847/1687] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:39.885305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:39.885305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:39.965870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59633326-470e-406c-b3ea-40e48c6f2e10 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:39.969708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:39.970512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:39.970800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:39.971071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:39.981830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:39.983288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:39.986235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f698-c7b7-4d19-9338-4d580bb2cd93 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:39.994954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 848/1688] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:39.995870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:39.997666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:40.002699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:40.008664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.009095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.009542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.022648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16416789-ef7c-42a3-b924-c41697cc5a64 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:40.028459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:40.030404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:40.038143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:40.039258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.039684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.040085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] PATH_INFO: `/v3/projects/b075bb5c53004a02a527e4e3ac0cddc4` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.043436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b075bb5c53004a02a527e4e3ac0cddc4)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:40.044849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 841/1689] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:40.060420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:40.060549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:40.060635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ba737-7e7e-44bd-8793-a9c7119cb921 service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:40.064563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bab3e-6d69-43aa-9dbf-6f3aa149119c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:40.065998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bab3e-6d69-43aa-9dbf-6f3aa149119c admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.067119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bab3e-6d69-43aa-9dbf-6f3aa149119c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.067494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bab3e-6d69-43aa-9dbf-6f3aa149119c admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.068465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bab3e-6d69-43aa-9dbf-6f3aa149119c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:40.073389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bab3e-6d69-43aa-9dbf-6f3aa149119c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:40.096698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 842/1690] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:40.107949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c82bc-d734-42f2-88fc-8a9a2c5ccbbf None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.108541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c82bc-d734-42f2-88fc-8a9a2c5ccbbf None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.118983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c82bc-d734-42f2-88fc-8a9a2c5ccbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.182369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-857c82bc-d734-42f2-88fc-8a9a2c5ccbbf None None] MFA Rules not processed for user `baac9b5e923747e6986720e3ac849056`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:40.202164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-857c82bc-d734-42f2-88fc-8a9a2c5ccbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:40.212094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 843/1691] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:40.214018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6c00-aa0b-4e29-acad-3f7f021e2921 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.215369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6c00-aa0b-4e29-acad-3f7f021e2921 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.215666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e6c00-aa0b-4e29-acad-3f7f021e2921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.243012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b075bb5c53004a02a527e4e3ac0cddc4', 'request_id': 'req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:40.250870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a90e6c00-aa0b-4e29-acad-3f7f021e2921 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:40.252406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a90e6c00-aa0b-4e29-acad-3f7f021e2921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:40.335309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 844/1692] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:40.336455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e6c00-aa0b-4e29-acad-3f7f021e2921 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:40.388299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b075bb5c53004a02a527e4e3ac0cddc4', 'request_id': 'req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:40.400342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b075bb5c53004a02a527e4e3ac0cddc4', 'request_id': 'req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:40.419989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] The token cache is being invalidate because project b075bb5c53004a02a527e4e3ac0cddc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:40.423059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:40.444267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:40.459328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 849/1693] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:39 2022] DELETE /identity/v3/projects/b075bb5c53004a02a527e4e3ac0cddc4 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:40.461597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:40.464494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:40.511126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e6c00-aa0b-4e29-acad-3f7f021e2921 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:40.543846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfb163-da36-453c-b2c9-a48512b2095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:40.544831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfb163-da36-453c-b2c9-a48512b2095a admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.545116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfb163-da36-453c-b2c9-a48512b2095a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.545426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfb163-da36-453c-b2c9-a48512b2095a admin admin] PATH_INFO: `/v3/projects/bf49b67827aa4e61beea42abc7fc24e6/users/01158a0ee3c240a3baab1960c0149a18/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.588795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfb163-da36-453c-b2c9-a48512b2095a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf49b67827aa4e61beea42abc7fc24e6, user_id=01158a0ee3c240a3baab1960c0149a18, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:40.594086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfb163-da36-453c-b2c9-a48512b2095a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:40.641530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe7ba8-9c59-4a90-8d3c-8444ae3723fb admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:40.646777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:40.648267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.648697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.649138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.651806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:40.719290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 845/1694] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:40 2022] PUT /identity/v3/projects/bf49b67827aa4e61beea42abc7fc24e6/users/01158a0ee3c240a3baab1960c0149a18/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:40.721549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:40.721959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:40.722227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:40.779101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:40.780422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:40.812365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:40.821387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:40.894371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 850/1695] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:40.894371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:40.895980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:40.951596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f1dfa7-f7dc-4cf8-bf34-dcb4217aa919 service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:40.988488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 846/1696] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:40.989554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:40.995213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:41.090337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54a7f0-e7ff-421c-b0b6-6e66db356b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:41.091032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a7f0-e7ff-421c-b0b6-6e66db356b6b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.092168 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a7f0-e7ff-421c-b0b6-6e66db356b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.092505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54a7f0-e7ff-421c-b0b6-6e66db356b6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.093279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54a7f0-e7ff-421c-b0b6-6e66db356b6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:41.119729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54a7f0-e7ff-421c-b0b6-6e66db356b6b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:41.147417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 851/1697] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:41.151124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d73f1-c69a-4a09-873a-588a86f4b118 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.152275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d73f1-c69a-4a09-873a-588a86f4b118 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.153395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d73f1-c69a-4a09-873a-588a86f4b118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.189455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4b0d73f1-c69a-4a09-873a-588a86f4b118 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:41.191590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b458df-59a4-4b37-85b0-337cfb4094f4 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:41.191755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4b0d73f1-c69a-4a09-873a-588a86f4b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:41.200612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 852/1698] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:41.201753 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d73f1-c69a-4a09-873a-588a86f4b118 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:41.204578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da65a356-d000-422b-8594-7a264a38f58f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:41.205548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a356-d000-422b-8594-7a264a38f58f service swift] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.207005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a356-d000-422b-8594-7a264a38f58f service swift] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.207511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da65a356-d000-422b-8594-7a264a38f58f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.218572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d73f1-c69a-4a09-873a-588a86f4b118 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:41.218679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-da65a356-d000-422b-8594-7a264a38f58f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:41.229669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:41.230992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.231934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.232352 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] PATH_INFO: `/v3/projects/bf49b67827aa4e61beea42abc7fc24e6/users/01158a0ee3c240a3baab1960c0149a18/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.238269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf49b67827aa4e61beea42abc7fc24e6, user_id=01158a0ee3c240a3baab1960c0149a18, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:41.243852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:41.268094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 853/1699] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:41 2022] PUT /identity/v3/projects/bf49b67827aa4e61beea42abc7fc24e6/users/01158a0ee3c240a3baab1960c0149a18/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:41.271365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:41.351145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a356-d000-422b-8594-7a264a38f58f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:41.353111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da65a356-d000-422b-8594-7a264a38f58f service swift] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:41.363156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 847/1700] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:13:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:13:41.364923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.365474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.367090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.468145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] MFA Rules not processed for user `01158a0ee3c240a3baab1960c0149a18`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:41.470542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:41.505646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f9992-7fe8-4f48-8cc2-f074631ae13b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:41.522687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:41.523535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.523672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.523934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.524674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:41.524997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:41.533497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:41.560771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:41.578607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 854/1701] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:41.583264 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:41.590007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:41.627524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8fb612-6065-4541-8db5-6493409c8b11 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:41.639657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:41.643556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.643556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.643556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] PATH_INFO: `/v3/users/e59aab502a9a43189637822d3dd2be57` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.645763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59aab502a9a43189637822d3dd2be57)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:41.647714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:41.724025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 848/1702] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:41.741437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:41.743383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:41.762955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad230e31-d3b8-444e-b653-bb9c167405c3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:41.775995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:41.777568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:41.778062 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:41.780461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] PATH_INFO: `/v3/projects/eb6faba45a60477e8202718b5cb2ca25` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:41.787650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb6faba45a60477e8202718b5cb2ca25)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:41.790258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:41.967267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59aab502a9a43189637822d3dd2be57'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:41.982058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb6faba45a60477e8202718b5cb2ca25', 'request_id': 'req-91d5e0ec-545b-4cd0-9c28-689db0b271a4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:41.985930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59aab502a9a43189637822d3dd2be57'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:41.988027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59aab502a9a43189637822d3dd2be57'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:42.009577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59aab502a9a43189637822d3dd2be57'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:42.168942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb6faba45a60477e8202718b5cb2ca25', 'request_id': 'req-91d5e0ec-545b-4cd0-9c28-689db0b271a4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:42.178278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb6faba45a60477e8202718b5cb2ca25', 'request_id': 'req-91d5e0ec-545b-4cd0-9c28-689db0b271a4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:42.204577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] The token cache is being invalidate because project eb6faba45a60477e8202718b5cb2ca25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:42.206649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:42.210377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:42.211565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:42.213725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 849/1703] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:41 2022] DELETE /identity/v3/projects/eb6faba45a60477e8202718b5cb2ca25 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:42.214736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:42.219530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:42.464181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5e0ec-545b-4cd0-9c28-689db0b271a4 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:42.464181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a195b6b-9c4e-43a5-bea7-864570c82f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:42.464181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195b6b-9c4e-43a5-bea7-864570c82f96 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:42.464181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195b6b-9c4e-43a5-bea7-864570c82f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:42.465892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a195b6b-9c4e-43a5-bea7-864570c82f96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:42.465892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a195b6b-9c4e-43a5-bea7-864570c82f96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:42.465892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a195b6b-9c4e-43a5-bea7-864570c82f96 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:42.557849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59aab502a9a43189637822d3dd2be57'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:42.568291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 855/1704] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:41 2022] DELETE /identity/v3/users/e59aab502a9a43189637822d3dd2be57 => generated 0 bytes in 989 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:42.569166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:42.572903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:42.655841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847f3b8-267e-46c4-916d-9b8afc2f412a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:42.660408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683240d0-0b50-4465-8779-141d56598927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:42.661220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683240d0-0b50-4465-8779-141d56598927 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:42.661542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683240d0-0b50-4465-8779-141d56598927 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:42.661823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683240d0-0b50-4465-8779-141d56598927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:42.663699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-683240d0-0b50-4465-8779-141d56598927 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:42.738374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 850/1705] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:42 2022] POST /identity/v3/projects => generated 363 bytes in 524 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:42.739544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3842e3-e7a4-4621-bb6d-5582eee22dff None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:42.739835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3842e3-e7a4-4621-bb6d-5582eee22dff None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:42.740115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3842e3-e7a4-4621-bb6d-5582eee22dff None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:42.742018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 851/1706] 10.222.0.35 () {54 vars in 912 bytes} [Tue May 24 10:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:42.742578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3842e3-e7a4-4621-bb6d-5582eee22dff None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:42.746580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7a3842e3-e7a4-4621-bb6d-5582eee22dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:42.793547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683240d0-0b50-4465-8779-141d56598927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:42.795190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683240d0-0b50-4465-8779-141d56598927 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:42.808098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 856/1707] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:42.809839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:42.810237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:42.810644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:42.834044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:42.835971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:42.978311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3842e3-e7a4-4621-bb6d-5582eee22dff None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:42.989548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:42.992584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:42.993120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:42.998407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] PATH_INFO: `/v3/users/14044d8c7fe0480088c3d06d90f91231` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:42.998407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] RBAC: Authorizing `identity:delete_user(user_id=14044d8c7fe0480088c3d06d90f91231)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:43.006161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:43.042082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 857/1708] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:43.043350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:43.045828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:43.067977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b78d5b-be68-4531-81d4-9b75729c6950 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:43.075256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca8bb0-fd87-4a27-8255-63c24c56361d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:43.076393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca8bb0-fd87-4a27-8255-63c24c56361d admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:43.076927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca8bb0-fd87-4a27-8255-63c24c56361d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:43.077493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca8bb0-fd87-4a27-8255-63c24c56361d admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:43.078474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca8bb0-fd87-4a27-8255-63c24c56361d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:43.081682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca8bb0-fd87-4a27-8255-63c24c56361d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:43.249774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 858/1709] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:43 2022] POST /identity/v3/users => generated 282 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:43.251325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:43.256246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:43.260683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:43.290635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] MFA Rules not processed for user `9937f1088b304574935fcdf579e4de7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:43.293052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:43.374528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 859/1710] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:43.382360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:43.382360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:43.396608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9702ba5-6a4f-4bf7-8db1-b77c418adb06 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:43.404242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:43.406174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:43.406933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:43.407819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] PATH_INFO: `/v3/users/335daaee431c47bc9f3274b4475f8963` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:43.409816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=335daaee431c47bc9f3274b4475f8963)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:43.413233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:43.441517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14044d8c7fe0480088c3d06d90f91231'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.453942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14044d8c7fe0480088c3d06d90f91231'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.470045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14044d8c7fe0480088c3d06d90f91231'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.471778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14044d8c7fe0480088c3d06d90f91231'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.495147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14044d8c7fe0480088c3d06d90f91231'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.693942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 852/1711] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:42 2022] DELETE /identity/v3/users/14044d8c7fe0480088c3d06d90f91231 => generated 0 bytes in 925 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:43.697650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:43.711595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:43.726692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335daaee431c47bc9f3274b4475f8963'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.746657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335daaee431c47bc9f3274b4475f8963'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.747518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335daaee431c47bc9f3274b4475f8963'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.780169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335daaee431c47bc9f3274b4475f8963'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.878842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff4c0d-057b-473c-8d85-9d4c4f5d591a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:43.885089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e2996e-893c-4392-8fc0-cfcb724967f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:43.885872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2996e-893c-4392-8fc0-cfcb724967f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:43.886212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2996e-893c-4392-8fc0-cfcb724967f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:43.886497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2996e-893c-4392-8fc0-cfcb724967f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:43.888915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d2e2996e-893c-4392-8fc0-cfcb724967f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:43.921725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335daaee431c47bc9f3274b4475f8963'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:43.942653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 860/1712] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:43 2022] DELETE /identity/v3/users/335daaee431c47bc9f3274b4475f8963 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:43.951305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:43.959191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:44.123490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2996e-893c-4392-8fc0-cfcb724967f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:44.125100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e2996e-893c-4392-8fc0-cfcb724967f4 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:44.132199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4bb66-ede0-4ee5-8203-b8b031ea0838 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:44.156858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:44.157696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.157696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.157903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.159008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:44.160425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:44.163348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 853/1713] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:44.165424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8322b-dfac-428b-98c3-676c37d976a6 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.165424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8322b-dfac-428b-98c3-676c37d976a6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.167141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8322b-dfac-428b-98c3-676c37d976a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.221195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 861/1714] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:44.221195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:44.221195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:44.234323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-45a8322b-dfac-428b-98c3-676c37d976a6 None None] MFA Rules not processed for user `9937f1088b304574935fcdf579e4de7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:44.236159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-45a8322b-dfac-428b-98c3-676c37d976a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:44.326481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 854/1715] 10.222.0.16 () {58 vars in 1015 bytes} [Tue May 24 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:44.327971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca17bef-56df-44c9-88a4-d438d137601d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.328260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca17bef-56df-44c9-88a4-d438d137601d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.328709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca17bef-56df-44c9-88a4-d438d137601d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.361847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1ca17bef-56df-44c9-88a4-d438d137601d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:44.363438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1ca17bef-56df-44c9-88a4-d438d137601d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:44.383724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 855/1716] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:44.385636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca17bef-56df-44c9-88a4-d438d137601d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:44.400388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca17bef-56df-44c9-88a4-d438d137601d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:44.408043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:44.409971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.410282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.411472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.414615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:44.419565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:44.436213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ad814-c6c0-475d-b080-b00e9bb055fe admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:44.440829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 856/1717] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:44.446726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:44.447448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.447716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:44.447833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.449031 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.450693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:44.454412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:44.471819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e6d2e-9745-4f57-953b-c310328cbc08 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:44.476227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:44.476996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.477299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.477602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.489325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:44.490268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:44.490801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:44.492754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:44.513448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 857/1718] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:44.513448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:44.516479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0919a9-4643-4d71-bf77-191d181f43a0 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:44.523499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32cf1c1-dda9-4330-86be-0d90b69637ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:44.524173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf1c1-dda9-4330-86be-0d90b69637ae service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.524462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf1c1-dda9-4330-86be-0d90b69637ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.524739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32cf1c1-dda9-4330-86be-0d90b69637ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.538209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 862/1719] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:44.540355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cf1c1-dda9-4330-86be-0d90b69637ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:44.542288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:44.542429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32cf1c1-dda9-4330-86be-0d90b69637ae service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:44.564307 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 858/1720] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:44.570682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd672c8e-4d36-4215-b8b1-80c06545b58b service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:44.579234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:44.580300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.580583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.581021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] PATH_INFO: `/v3/projects/29b3588f70224b428ab2976865d0b169/users/761aa53f47854458b37504bcf29c280b/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.633207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b3588f70224b428ab2976865d0b169, user_id=761aa53f47854458b37504bcf29c280b, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:44.650659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:44.761872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 863/1721] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:44 2022] PUT /identity/v3/projects/29b3588f70224b428ab2976865d0b169/users/761aa53f47854458b37504bcf29c280b/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:44.774165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73314b-969e-46e6-bb61-d71665ba9b4b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:44.774632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73314b-969e-46e6-bb61-d71665ba9b4b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:44.775112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73314b-969e-46e6-bb61-d71665ba9b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:44.798249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:44.828096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-fa73314b-969e-46e6-bb61-d71665ba9b4b None None] MFA Rules not processed for user `761aa53f47854458b37504bcf29c280b`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:44.829989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fa73314b-969e-46e6-bb61-d71665ba9b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:44.936672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 859/1722] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:44.941719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:45.064351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96214182-7db5-447f-9ab1-5908841ce6e6 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:45.070556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:45.071171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:45.071634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:45.071793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:45.072678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:45.074553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:45.111087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:45.282766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 864/1723] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:44 2022] POST /identity/v3/projects => generated 365 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:45.283474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73314b-969e-46e6-bb61-d71665ba9b4b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:45.311879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73314b-969e-46e6-bb61-d71665ba9b4b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:45.479468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:45.480037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:45.480362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:45.480591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:45.481479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:45.490830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:45.490970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:45.490970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:45.509918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753affe0-d5e3-4d01-b1fe-c5cdb5278df8 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:45.570787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:45.571698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:45.571985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:45.572448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] PATH_INFO: `/v3/projects/5de3661c07d34c4fbf2f27dd3b4cc0cf` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:45.575173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de3661c07d34c4fbf2f27dd3b4cc0cf)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:45.579444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:45.644525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 860/1724] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:45 2022] POST /identity/v3/users => generated 289 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:45.646677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:45.681903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f06db1-d850-4280-af28-2cdb809ee870 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:45.711881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:45.713482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:45.713774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:45.714058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:45.714839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:45.728752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:45.781305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 861/1725] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:45.787844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:45.847383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38b9dc-5cc2-497b-af5a-7274ef1c7389 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:45.876642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de3661c07d34c4fbf2f27dd3b4cc0cf', 'request_id': 'req-d9f9169b-e73d-43ee-9189-0ec2d14809ee'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:46.019732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:46.021961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de3661c07d34c4fbf2f27dd3b4cc0cf', 'request_id': 'req-d9f9169b-e73d-43ee-9189-0ec2d14809ee'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:46.024153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.025008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.025535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] PATH_INFO: `/v3/projects/f02061f380ab44f7a9964dc620ec79f5/users/547309cea5b947148d7eb83abf459db3/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.033218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de3661c07d34c4fbf2f27dd3b4cc0cf', 'request_id': 'req-d9f9169b-e73d-43ee-9189-0ec2d14809ee'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:46.068199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02061f380ab44f7a9964dc620ec79f5, user_id=547309cea5b947148d7eb83abf459db3, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:46.079873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] The token cache is being invalidate because project 5de3661c07d34c4fbf2f27dd3b4cc0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:46.080215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:46.082490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d9f9169b-e73d-43ee-9189-0ec2d14809ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:46.085790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 865/1726] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:45 2022] DELETE /identity/v3/projects/5de3661c07d34c4fbf2f27dd3b4cc0cf => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:46.092055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:46.094786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c302a-bcea-4b0b-a53e-2d42cf03d5c6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.100482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c302a-bcea-4b0b-a53e-2d42cf03d5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.100482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c302a-bcea-4b0b-a53e-2d42cf03d5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.169308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-576c302a-bcea-4b0b-a53e-2d42cf03d5c6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:46.169308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-576c302a-bcea-4b0b-a53e-2d42cf03d5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:46.200217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 866/1727] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:46.202653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5c550-ca05-4813-9112-afe743b862b7 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.202950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5c550-ca05-4813-9112-afe743b862b7 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.203268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5c550-ca05-4813-9112-afe743b862b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.247504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 862/1728] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:45 2022] PUT /identity/v3/projects/f02061f380ab44f7a9964dc620ec79f5/users/547309cea5b947148d7eb83abf459db3/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:46.249696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-96e5c550-ca05-4813-9112-afe743b862b7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:46.251427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-96e5c550-ca05-4813-9112-afe743b862b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:46.252215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:46.255648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:46.498983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a2bfe-3557-47f6-a7a0-95b94bcaac99 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:46.514412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2b73f5-d670-4c9f-a4df-64da1289f845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:46.528022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b73f5-d670-4c9f-a4df-64da1289f845 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.529802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b73f5-d670-4c9f-a4df-64da1289f845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.530986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2b73f5-d670-4c9f-a4df-64da1289f845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.532325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b73f5-d670-4c9f-a4df-64da1289f845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:46.535352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2b73f5-d670-4c9f-a4df-64da1289f845 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:46.582786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 863/1729] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:46.591964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 867/1730] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:46.592184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d23e-bb9a-4de8-9172-17df1a617c1e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.592184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d23e-bb9a-4de8-9172-17df1a617c1e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.592184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d23e-bb9a-4de8-9172-17df1a617c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.593631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5c550-ca05-4813-9112-afe743b862b7 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:46.616835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5c550-ca05-4813-9112-afe743b862b7 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:46.627219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-40a5d23e-bb9a-4de8-9172-17df1a617c1e None None] MFA Rules not processed for user `baac9b5e923747e6986720e3ac849056`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:46.628752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-40a5d23e-bb9a-4de8-9172-17df1a617c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:46.632473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:46.633565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.633929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.634287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.638052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:46.638463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:46.645477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:46.684488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 868/1731] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:46.690772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:46.715891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c2ff9-1267-400e-8355-a6cd2411b017 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:46.729720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:46.731315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.731974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.732567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] PATH_INFO: `/v3/projects/f02061f380ab44f7a9964dc620ec79f5/users/547309cea5b947148d7eb83abf459db3/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.735863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02061f380ab44f7a9964dc620ec79f5, user_id=547309cea5b947148d7eb83abf459db3, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:46.746678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:46.784128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 864/1732] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:46.790504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5d23e-bb9a-4de8-9172-17df1a617c1e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:46.814410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 869/1733] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:46 2022] PUT /identity/v3/projects/f02061f380ab44f7a9964dc620ec79f5/users/547309cea5b947148d7eb83abf459db3/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:46.815932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:46.822308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:46.839015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5d23e-bb9a-4de8-9172-17df1a617c1e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:46.991056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929184fe-2d0e-446c-9718-e229fbdeedd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:46.992028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929184fe-2d0e-446c-9718-e229fbdeedd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:46.992373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929184fe-2d0e-446c-9718-e229fbdeedd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:46.992753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929184fe-2d0e-446c-9718-e229fbdeedd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:46.993904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929184fe-2d0e-446c-9718-e229fbdeedd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:46.996171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929184fe-2d0e-446c-9718-e229fbdeedd2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:47.014721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e37913-602b-4baf-8550-dbb96e4b5a59 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:47.038519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:47.042454 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:47.044284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:47.044573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] PATH_INFO: `/v3/projects/35a9ad5293a844f4a481f73dcee4f86b` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:47.056023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a9ad5293a844f4a481f73dcee4f86b)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:47.056023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:47.182341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 865/1734] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:46 2022] POST /identity/v3/projects => generated 387 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:47.188305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5543fd-b667-4dda-81ea-ae2e66fed7e8 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:47.188848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5543fd-b667-4dda-81ea-ae2e66fed7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:47.189140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5543fd-b667-4dda-81ea-ae2e66fed7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:47.255271 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ff5543fd-b667-4dda-81ea-ae2e66fed7e8 None None] MFA Rules not processed for user `547309cea5b947148d7eb83abf459db3`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:47.256520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ff5543fd-b667-4dda-81ea-ae2e66fed7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:47.304456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a9ad5293a844f4a481f73dcee4f86b', 'request_id': 'req-439a4e28-64e6-4577-89a6-544973f48685'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:47.408934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:47.438763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a9ad5293a844f4a481f73dcee4f86b', 'request_id': 'req-439a4e28-64e6-4577-89a6-544973f48685'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:47.446879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a9ad5293a844f4a481f73dcee4f86b', 'request_id': 'req-439a4e28-64e6-4577-89a6-544973f48685'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:47.468966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 866/1735] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:47 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:47.476201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:47.476625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:47.477195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:47.489636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] The token cache is being invalidate because project 35a9ad5293a844f4a481f73dcee4f86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:47.489801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:47.494728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:47.501438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 870/1736] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:46 2022] DELETE /identity/v3/projects/35a9ad5293a844f4a481f73dcee4f86b => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:47.505155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:47.506995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:47.507318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] MFA Rules not processed for user `7bffd469507b4dbc8d3146f051c8c8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:47.508791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:47.653462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 867/1737] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:47.654571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:47.661867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:47.672970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439a4e28-64e6-4577-89a6-544973f48685 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:47.723962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:47.731259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d94d87-9ece-443c-9229-d0c90c8d0abc None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:47.733591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:47.736108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:47.740500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] PATH_INFO: `/v3/projects/4ffb58b3629f4ed3953f34bf901624c7` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:47.759242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ffb58b3629f4ed3953f34bf901624c7)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:47.769196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:47.771987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:47.773672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:47.774532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:47.777637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:47.781213 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:47.790766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:47.960722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 868/1738] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:47 2022] POST /identity/v3/users => generated 294 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:47.972631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:47.977931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:48.027495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ffb58b3629f4ed3953f34bf901624c7', 'request_id': 'req-f9f76f84-3575-41df-9f63-bcfeccf1a35e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.100971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ffb58b3629f4ed3953f34bf901624c7', 'request_id': 'req-f9f76f84-3575-41df-9f63-bcfeccf1a35e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.110996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ffb58b3629f4ed3953f34bf901624c7', 'request_id': 'req-f9f76f84-3575-41df-9f63-bcfeccf1a35e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.149253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] The token cache is being invalidate because project 4ffb58b3629f4ed3953f34bf901624c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:48.150682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.152990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f9f76f84-3575-41df-9f63-bcfeccf1a35e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.160841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 871/1739] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:47 2022] DELETE /identity/v3/projects/4ffb58b3629f4ed3953f34bf901624c7 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:48.163802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516daca-f391-495b-a2b5-1cd14a7f9950 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.164099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516daca-f391-495b-a2b5-1cd14a7f9950 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.177290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516daca-f391-495b-a2b5-1cd14a7f9950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.178351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f90f2-5c7b-438b-be2a-d7ce7f28a198 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:48.199157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:48.242267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5516daca-f391-495b-a2b5-1cd14a7f9950 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:48.244700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5516daca-f391-495b-a2b5-1cd14a7f9950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:48.262157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:48.263563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.264644 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.265247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.271147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:48.517376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 872/1740] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:48.520009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b120962-fdae-41af-ae20-b6f8846379b5 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.520418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b120962-fdae-41af-ae20-b6f8846379b5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.520710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b120962-fdae-41af-ae20-b6f8846379b5 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.525292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 873/1741] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:13:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:48.526232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b120962-fdae-41af-ae20-b6f8846379b5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:48.529300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4b120962-fdae-41af-ae20-b6f8846379b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:48.576513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b120962-fdae-41af-ae20-b6f8846379b5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:48.591039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d00a114-bf3b-47cb-ab9e-3a951e18b39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:48.591941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a114-bf3b-47cb-ab9e-3a951e18b39d admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.592490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a114-bf3b-47cb-ab9e-3a951e18b39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.592780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d00a114-bf3b-47cb-ab9e-3a951e18b39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.593675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00a114-bf3b-47cb-ab9e-3a951e18b39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:48.595331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d00a114-bf3b-47cb-ab9e-3a951e18b39d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:48.599871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:48.602757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:48.648700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 869/1742] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:47 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:48.653859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:48.667269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 874/1743] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:48.671817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6461406-8401-437d-be84-0905a493aa9c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.672471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6461406-8401-437d-be84-0905a493aa9c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.672766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6461406-8401-437d-be84-0905a493aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.673406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:48.695624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1828a0-aecf-4a67-8c22-105e4f71e798 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:48.704216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:48.704216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.704216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.704216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] PATH_INFO: `/v3/users/01158a0ee3c240a3baab1960c0149a18` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.704619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01158a0ee3c240a3baab1960c0149a18)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:48.708654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:48.733936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c6461406-8401-437d-be84-0905a493aa9c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:48.735198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c6461406-8401-437d-be84-0905a493aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:48.750774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 875/1744] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:48.758160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6461406-8401-437d-be84-0905a493aa9c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:48.765582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6461406-8401-437d-be84-0905a493aa9c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:48.775688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cc03a-a80d-4b5c-a330-287f9352f9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:48.776553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc03a-a80d-4b5c-a330-287f9352f9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.777072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc03a-a80d-4b5c-a330-287f9352f9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.777540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cc03a-a80d-4b5c-a330-287f9352f9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.778935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cc03a-a80d-4b5c-a330-287f9352f9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:48.782005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cc03a-a80d-4b5c-a330-287f9352f9ca admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:48.799518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 876/1745] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:48.801771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:48.802074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:48.802404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:48.837081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:48.839045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:48.935087 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01158a0ee3c240a3baab1960c0149a18'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.937313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01158a0ee3c240a3baab1960c0149a18'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.953658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01158a0ee3c240a3baab1960c0149a18'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.956462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01158a0ee3c240a3baab1960c0149a18'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:48.988388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01158a0ee3c240a3baab1960c0149a18'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:49.024326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 877/1746] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:49.026010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:49.027725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:49.050385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d2ab2-a900-4f16-a8bc-de0bbd5c2f45 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:49.057151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:49.076806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:49.076806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:49.076806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:49.076806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:49.084515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:49.086399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:49.119043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 878/1747] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:49.122729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:49.125449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:49.152307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7f2beb-d05e-4a98-867d-c66af42d920a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:49.161139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:49.167949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 870/1748] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:48 2022] DELETE /identity/v3/users/01158a0ee3c240a3baab1960c0149a18 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:49.168963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:49.171292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:49.194097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:49.195628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:49.196202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] PATH_INFO: `/v3/projects/c378195158134bdab763427401838ace/users/234d12247c23469aa3710a6aece8317a/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:49.295390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c378195158134bdab763427401838ace, user_id=234d12247c23469aa3710a6aece8317a, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:49.306318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:49.340733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f85da-7af2-47d9-8dec-092c8c6efb12 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:49.358049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:49.360394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:49.360918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:49.369846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:49.369846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:49.413037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 879/1749] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:49 2022] PUT /identity/v3/projects/c378195158134bdab763427401838ace/users/234d12247c23469aa3710a6aece8317a/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:49.415563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:49.420671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:49.586834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0e766-50bd-4358-b2ee-00261f5603ef admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:49.588631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:49.590764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:49.596651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceddc376-1de6-4d7f-9612-cc03ee418415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:49.597379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc376-1de6-4d7f-9612-cc03ee418415 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:49.597665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc376-1de6-4d7f-9612-cc03ee418415 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:49.597945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceddc376-1de6-4d7f-9612-cc03ee418415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:49.613482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 871/1750] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:49 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:49.614559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:49.616809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddc376-1de6-4d7f-9612-cc03ee418415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:49.618415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceddc376-1de6-4d7f-9612-cc03ee418415 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:49.643822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 880/1751] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:49.645493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:49.645785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:49.646065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:49.690786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] MFA Rules not processed for user `234d12247c23469aa3710a6aece8317a`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:49.692240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:49.875632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 881/1752] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:49.879879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:49.883511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f8cae-80d7-4768-ae0d-6bb224af6efd service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:49.885864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:50.035170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:50.037753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:50.038056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:50.038340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:50.039058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:50.062675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:50.109317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a3d9-7b5d-4c80-b414-226727c533ed None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:50.116560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:50.117705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:50.118023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:50.118427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] PATH_INFO: `/v3/users/7360414f56854b2995b3f78eef666430` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:50.119952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7360414f56854b2995b3f78eef666430)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:50.122385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:50.216856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 872/1753] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:49 2022] POST /identity/v3/projects => generated 403 bytes in 603 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:50.217731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:50.368102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31457c7c-66a6-49af-bc31-bbddedcd7faf admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:50.375534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:50.378336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:50.379604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:50.380470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:50.381706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:50.384273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7360414f56854b2995b3f78eef666430'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:50.387497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:50.401381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7360414f56854b2995b3f78eef666430'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:50.403097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7360414f56854b2995b3f78eef666430'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:50.436488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7360414f56854b2995b3f78eef666430'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:50.538206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7360414f56854b2995b3f78eef666430'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:50.547822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 882/1754] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:49 2022] DELETE /identity/v3/users/7360414f56854b2995b3f78eef666430 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:50.548838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:50.552759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 873/1755] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:50 2022] POST /identity/v3/projects => generated 387 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:50.553163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:50.553935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:50.556047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:50.646852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a7154-1774-4a9d-a5cd-8d047cd317ec admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:50.651650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b913041-1a39-4447-9721-80d660428c7c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:50.653744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8c77c-b585-4de5-b55a-39ea4a08b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:50.657841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8c77c-b585-4de5-b55a-39ea4a08b0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:50.657841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8c77c-b585-4de5-b55a-39ea4a08b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:50.657841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8c77c-b585-4de5-b55a-39ea4a08b0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:50.657841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8c77c-b585-4de5-b55a-39ea4a08b0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:50.658178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:50.658178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:50.658357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c8c77c-b585-4de5-b55a-39ea4a08b0d6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:50.658443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:50.658443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] PATH_INFO: `/v3/projects/bf49b67827aa4e61beea42abc7fc24e6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:50.660524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf49b67827aa4e61beea42abc7fc24e6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:50.662628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:50.787997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 883/1756] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:50 2022] POST /identity/v3/users => generated 302 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:50.792637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:50.792637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:50.792637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:50.819308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:50.848016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf49b67827aa4e61beea42abc7fc24e6', 'request_id': 'req-01af5858-079f-41e4-a788-968ad1d120d0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:50.856221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:50.857542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:50.931320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 884/1757] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:50.932721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:50.934254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:50.950229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b284465-92a1-45bf-99c1-fa68a303f68d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:50.957948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d31113-91b7-41a8-a52e-c16d94bf0776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:50.958766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d31113-91b7-41a8-a52e-c16d94bf0776 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:50.959051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d31113-91b7-41a8-a52e-c16d94bf0776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:50.959616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d31113-91b7-41a8-a52e-c16d94bf0776 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:50.960400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d31113-91b7-41a8-a52e-c16d94bf0776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:50.962689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d31113-91b7-41a8-a52e-c16d94bf0776 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:50.973973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf49b67827aa4e61beea42abc7fc24e6', 'request_id': 'req-01af5858-079f-41e4-a788-968ad1d120d0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:50.982378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf49b67827aa4e61beea42abc7fc24e6', 'request_id': 'req-01af5858-079f-41e4-a788-968ad1d120d0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:51.002505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] The token cache is being invalidate because project bf49b67827aa4e61beea42abc7fc24e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:51.003399 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:51.004286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:51.005909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 874/1758] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:50 2022] DELETE /identity/v3/projects/bf49b67827aa4e61beea42abc7fc24e6 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:51.009129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.013484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:51.037136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af5858-079f-41e4-a788-968ad1d120d0 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.042064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.042744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.043085 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.043367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.044156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.045830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.061447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 875/1759] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:51.061447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.062033 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:51.127122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc46711-f486-4792-a8b4-e20b9d816da9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.131609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.132374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.132639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.132918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.134983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:51.143954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 885/1760] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:50 2022] POST /identity/v3/users => generated 300 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:51.143954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.143954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.143954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.149563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.151018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.173535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:51.175228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:51.179073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 876/1761] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:51.180074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.202849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 886/1762] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:51.203943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf3884-096c-4ee6-a2a1-97e6c116dc86 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.212614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.212614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:51.212844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a506cb-0228-4963-bcdc-6b2e9884f318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.212844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a506cb-0228-4963-bcdc-6b2e9884f318 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.212844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a506cb-0228-4963-bcdc-6b2e9884f318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.212844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a506cb-0228-4963-bcdc-6b2e9884f318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.213156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a506cb-0228-4963-bcdc-6b2e9884f318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.219141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a506cb-0228-4963-bcdc-6b2e9884f318 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.223903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17638c80-59e3-4f5e-bd4b-675b6ac63886 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.230537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.231499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.231929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.232353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.233246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.235598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.242734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 877/1763] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:51.244239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b912-678d-46b7-a5cc-f36a167fb24e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.244535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b912-678d-46b7-a5cc-f36a167fb24e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.244883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2b912-678d-46b7-a5cc-f36a167fb24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.252508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 887/1764] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:51.254400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.267292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f76b36-0226-4d94-bcb1-4e0fff1b0cca admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.274658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.276968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.277486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.278076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] PATH_INFO: `/v3/projects/ee7a9efc0b0d4a61a710ce9d2465c055/users/c4e16022c3884534aa1070c46a3fe650/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.300888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee7a9efc0b0d4a61a710ce9d2465c055, user_id=c4e16022c3884534aa1070c46a3fe650, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.304673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.311543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:51.315373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0bf2b912-678d-46b7-a5cc-f36a167fb24e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:51.324835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0bf2b912-678d-46b7-a5cc-f36a167fb24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:51.337767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 878/1765] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:51.345562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b912-678d-46b7-a5cc-f36a167fb24e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.357522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2b912-678d-46b7-a5cc-f36a167fb24e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.363089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1068c80b-0667-4fa0-ae09-b8c8e32fdab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.364358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068c80b-0667-4fa0-ae09-b8c8e32fdab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.364724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068c80b-0667-4fa0-ae09-b8c8e32fdab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.365008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1068c80b-0667-4fa0-ae09-b8c8e32fdab5 admin admin] PATH_INFO: `/v3/projects/c251bfe6e28f47cba472557fca2e2c2d/users/f27eccd6c9e246b297e1b14ee8ebb055/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.381825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068c80b-0667-4fa0-ae09-b8c8e32fdab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c251bfe6e28f47cba472557fca2e2c2d, user_id=f27eccd6c9e246b297e1b14ee8ebb055, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.384626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1068c80b-0667-4fa0-ae09-b8c8e32fdab5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.511323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 888/1766] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:51 2022] PUT /identity/v3/projects/ee7a9efc0b0d4a61a710ce9d2465c055/users/c4e16022c3884534aa1070c46a3fe650/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:51.511323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.529149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 879/1767] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:51 2022] PUT /identity/v3/projects/c251bfe6e28f47cba472557fca2e2c2d/users/f27eccd6c9e246b297e1b14ee8ebb055/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:51.530662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074fc33-55b7-4a32-bb48-ddd3e949270b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.535074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074fc33-55b7-4a32-bb48-ddd3e949270b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.535074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074fc33-55b7-4a32-bb48-ddd3e949270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.611397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4074fc33-55b7-4a32-bb48-ddd3e949270b None None] MFA Rules not processed for user `c4e16022c3884534aa1070c46a3fe650`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:51.613201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4074fc33-55b7-4a32-bb48-ddd3e949270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:51.661230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f65992-5e35-4a05-b297-44c4c135e479 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.772771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 880/1768] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:51.776749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074fc33-55b7-4a32-bb48-ddd3e949270b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.780630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.782746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.783255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.787389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.788214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:51.788532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.794697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.802063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074fc33-55b7-4a32-bb48-ddd3e949270b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.807013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.807694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.807976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.808263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.809052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.810703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.814094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 889/1769] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:51.814979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.822975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 881/1770] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:51.833230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:51.833372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ac24e-305a-4d5f-886c-30ecec199005 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.833372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.833590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.833755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.834034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.834791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.835958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911623a-f816-45c7-928f-348ecc5e66ae admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:51.837235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.841301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:51.841999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:51.842287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:51.842560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:51.843258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:51.845152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:51.946913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 882/1771] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:51 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:51.947791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:52.019456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 890/1772] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:51 2022] POST /identity/v3/projects => generated 403 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:52.020875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:52.023628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:52.106549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba346584-d2b3-4f47-823f-531ebceac34b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:52.132498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c80660-71a6-41b0-a786-fa1d632c09ea admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:52.139525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:52.143444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:52.143444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:52.143444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] PATH_INFO: `/v3/projects/82d4fd44d8fe4c9b888e417997f4da03` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:52.152731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82d4fd44d8fe4c9b888e417997f4da03)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:52.167984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:52.284808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:52.287203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:52.288646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:52.289147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] PATH_INFO: `/v3/projects/c251bfe6e28f47cba472557fca2e2c2d/users/f27eccd6c9e246b297e1b14ee8ebb055/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:52.293583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c251bfe6e28f47cba472557fca2e2c2d, user_id=f27eccd6c9e246b297e1b14ee8ebb055, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:52.300025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:52.324750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 883/1773] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:51 2022] PUT /identity/v3/projects/c251bfe6e28f47cba472557fca2e2c2d/users/f27eccd6c9e246b297e1b14ee8ebb055/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:52.324750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:52.384347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82d4fd44d8fe4c9b888e417997f4da03', 'request_id': 'req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:52.473569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57524c65-c9a8-44cc-baa0-98babbc064d1 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:52.551935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82d4fd44d8fe4c9b888e417997f4da03', 'request_id': 'req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:52.559589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82d4fd44d8fe4c9b888e417997f4da03', 'request_id': 'req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:52.583455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:52.585536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d5608a-5a31-4ae2-96e1-8417db80fb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:52.586232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5608a-5a31-4ae2-96e1-8417db80fb73 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:52.586514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5608a-5a31-4ae2-96e1-8417db80fb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:52.589869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d5608a-5a31-4ae2-96e1-8417db80fb73 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:52.590622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5608a-5a31-4ae2-96e1-8417db80fb73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:52.595468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d5608a-5a31-4ae2-96e1-8417db80fb73 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:52.598329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] The token cache is being invalidate because project 82d4fd44d8fe4c9b888e417997f4da03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:52.598622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:52.601788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:52.606980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 891/1774] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:52 2022] DELETE /identity/v3/projects/82d4fd44d8fe4c9b888e417997f4da03 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:52.611062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:52.623331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:52.657208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05b9c0-5dc6-4c9a-9f26-57fc3bfd442e admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:52.668303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:52.672441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:52.673024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:52.673942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:52.676511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:52.678999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:52.737010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 884/1775] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:52 2022] POST /identity/v3/users => generated 293 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:52.738681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:52.739172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:52.739623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:52.818894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] MFA Rules not processed for user `f27eccd6c9e246b297e1b14ee8ebb055`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:52.820355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:52.853177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 892/1776] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:52 2022] POST /identity/v3/users => generated 308 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:52.860305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:52.863052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:52.901125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1c529-9b23-438a-81c6-92acacb075f4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:52.914622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:52.915704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:52.916058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:52.916338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] PATH_INFO: `/v3/users/8a8e9ccbef474093861aaf61ed5ac37d` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:52.918023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a8e9ccbef474093861aaf61ed5ac37d)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:52.920257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:52.959825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 885/1777] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:52.960985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:52.962451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:53.009183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f644c-80e0-4aff-8a44-1c77fed051ab None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.024715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.025950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.026238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.026514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.027287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.028913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.053258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 886/1778] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:53.054597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:53.067878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b48a9-a5dd-473f-92f8-b96755084031 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.074108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.075408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.075923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.076400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.077387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.080477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.093945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 887/1779] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:53.095776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:53.099085 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:53.188557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e852e380-8782-406b-8540-89167bcd30bc admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.194612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.195567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.195975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.196295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.204646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a8e9ccbef474093861aaf61ed5ac37d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:53.208741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.211354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.223278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a8e9ccbef474093861aaf61ed5ac37d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:53.224079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a8e9ccbef474093861aaf61ed5ac37d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:53.225398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 888/1780] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:53.230417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:53.250930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cbb8b-689d-466c-819f-412028dce12a service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.258361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.259054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.259385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.259682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.260550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.262296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.282642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 889/1781] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:53.284851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:53.301613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a8e9ccbef474093861aaf61ed5ac37d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:53.301815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37813da-a43f-4282-a61f-0ac188f635fb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.315862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.317571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.318151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.318693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] PATH_INFO: `/v3/projects/2255de4951a54bf1bdbdb04946f51014/users/56865cfb20f9489dacc6a8b60bac4bb9/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.348242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2255de4951a54bf1bdbdb04946f51014, user_id=56865cfb20f9489dacc6a8b60bac4bb9, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.351420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.426079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a8e9ccbef474093861aaf61ed5ac37d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:53.433197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 893/1782] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:52 2022] DELETE /identity/v3/users/8a8e9ccbef474093861aaf61ed5ac37d => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:53.446335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:53.446335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:53.577737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 890/1783] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:53 2022] PUT /identity/v3/projects/2255de4951a54bf1bdbdb04946f51014/users/56865cfb20f9489dacc6a8b60bac4bb9/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:53.579733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81cd8-95bf-4045-9022-7f6f81e4afb4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.579826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:53.583881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:53.644613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506bd2b2-ef21-4714-95d5-8ec478923197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.645598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bd2b2-ef21-4714-95d5-8ec478923197 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.645892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bd2b2-ef21-4714-95d5-8ec478923197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.646190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bd2b2-ef21-4714-95d5-8ec478923197 admin admin] PATH_INFO: `/v3/projects/a2b7a182f2e94b3a9e1ca2d88ac23206/users/9fd4409ec039457c9bd3a1eb3306629e/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.678104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bd2b2-ef21-4714-95d5-8ec478923197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b7a182f2e94b3a9e1ca2d88ac23206, user_id=9fd4409ec039457c9bd3a1eb3306629e, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.680802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506bd2b2-ef21-4714-95d5-8ec478923197 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.742801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4de4bd-f2e3-4dfb-a589-67e6955d7ea0 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.748411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.749075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.749596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.749891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.750779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.756990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.768350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 891/1784] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:53.772615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:53.797560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b4e20-9c55-4227-87ed-a4e0e55c3b64 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:53.804727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1319f-0b77-4a80-a22f-4c897d34d0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:53.805759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1319f-0b77-4a80-a22f-4c897d34d0ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.806005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1319f-0b77-4a80-a22f-4c897d34d0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.806287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1319f-0b77-4a80-a22f-4c897d34d0ac admin admin] PATH_INFO: `/v3/projects/2255de4951a54bf1bdbdb04946f51014/users/56865cfb20f9489dacc6a8b60bac4bb9/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.808366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1319f-0b77-4a80-a22f-4c897d34d0ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=2255de4951a54bf1bdbdb04946f51014, user_id=56865cfb20f9489dacc6a8b60bac4bb9, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:53.811365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1319f-0b77-4a80-a22f-4c897d34d0ac admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:53.828402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 892/1785] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:53 2022] PUT /identity/v3/projects/2255de4951a54bf1bdbdb04946f51014/users/56865cfb20f9489dacc6a8b60bac4bb9/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:13:53.839051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:53.839345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:53.839628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:53.897967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] MFA Rules not processed for user `56865cfb20f9489dacc6a8b60bac4bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:53.899843 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:54.111026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 893/1786] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:53 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:13:54.111026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:54.111026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:54.240222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa0b97-f737-404c-82a1-57173b762ba9 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:54.244501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:54.251493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:54.251898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:54.252229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:54.268238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:54.269740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:54.279506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 894/1787] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:54 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:13:54.440608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:54.502656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d79421-d116-403b-b861-e24eb5dfd6a3 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:54.507156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:54.508022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:54.508298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:54.508576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] PATH_INFO: `/v3/users/c4e16022c3884534aa1070c46a3fe650` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:54.509768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e16022c3884534aa1070c46a3fe650)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:54.511676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:54.779852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 894/1788] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:53 2022] PUT /identity/v3/projects/a2b7a182f2e94b3a9e1ca2d88ac23206/users/9fd4409ec039457c9bd3a1eb3306629e/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 1333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:54.779852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627373a-b2db-412c-aaca-8a458997effb None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:54.779852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627373a-b2db-412c-aaca-8a458997effb None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:54.780504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627373a-b2db-412c-aaca-8a458997effb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:54.813300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7627373a-b2db-412c-aaca-8a458997effb None None] MFA Rules not processed for user `9fd4409ec039457c9bd3a1eb3306629e`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:54.814697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7627373a-b2db-412c-aaca-8a458997effb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:54.898047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 895/1789] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:54.902494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627373a-b2db-412c-aaca-8a458997effb None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:54.973470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627373a-b2db-412c-aaca-8a458997effb None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:54.978615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:54.979476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:54.979913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:54.980032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:54.980726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:54.983032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:55.013758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e16022c3884534aa1070c46a3fe650'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.015513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e16022c3884534aa1070c46a3fe650'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.025101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e16022c3884534aa1070c46a3fe650'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.035056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e16022c3884534aa1070c46a3fe650'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.049549 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27d9b5b9-234c-4c13-860b-31b58ef51c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e16022c3884534aa1070c46a3fe650'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.097096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 896/1790] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:54 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:13:55.101710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:55.105114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:55.182525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 895/1791] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:54 2022] DELETE /identity/v3/users/c4e16022c3884534aa1070c46a3fe650 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:13:55.191026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.191569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.191933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.248074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:55.249490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:55.250041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d355d8-23a6-47a5-81af-2383e130be70 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:55.336521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 896/1792] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:55.337626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:55.342244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:55.372665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:55.373571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed0e2c-fe06-4290-9adc-d27088b133ff None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:55.374865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.375763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.378114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.380815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:55.381165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:55.397790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.397886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:55.397975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.397975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] PATH_INFO: `/v3/projects/7cc5b120f5b6434b89a7686298bea31c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.397975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc5b120f5b6434b89a7686298bea31c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:55.397975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:55.568252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 897/1793] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:55 2022] POST /identity/v3/users => generated 300 bytes in 467 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:55.570879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:55.572690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:55.623641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc5b120f5b6434b89a7686298bea31c', 'request_id': 'req-59a53def-8856-4e2b-bdd1-564a664b1677'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.636091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd6539-6c09-4dd7-8b68-78166286ac71 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:55.686170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:55.686832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.687165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.687439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.715994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc5b120f5b6434b89a7686298bea31c', 'request_id': 'req-59a53def-8856-4e2b-bdd1-564a664b1677'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.726738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc5b120f5b6434b89a7686298bea31c', 'request_id': 'req-59a53def-8856-4e2b-bdd1-564a664b1677'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.745353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] The token cache is being invalidate because project 7cc5b120f5b6434b89a7686298bea31c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:55.745935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.746796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:55.748572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 897/1794] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:55 2022] DELETE /identity/v3/projects/7cc5b120f5b6434b89a7686298bea31c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:55.749660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:55.751498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:55.777902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:55.779347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:55.790170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 898/1795] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:55.791033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:55.806912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfae12f-7962-4f5e-bfc9-b49f4163dbca service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:55.819919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3268ca-7d52-4795-a26f-6d2f5d63cc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:55.821838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3268ca-7d52-4795-a26f-6d2f5d63cc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.822941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3268ca-7d52-4795-a26f-6d2f5d63cc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.823710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3268ca-7d52-4795-a26f-6d2f5d63cc2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.825004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3268ca-7d52-4795-a26f-6d2f5d63cc2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:55.828359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3268ca-7d52-4795-a26f-6d2f5d63cc2e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:55.843265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a53def-8856-4e2b-bdd1-564a664b1677 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:55.844313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 899/1796] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:55.846175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.846728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.847233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.852015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:55.853030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.853450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.853819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] PATH_INFO: `/v3/users/761aa53f47854458b37504bcf29c280b` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.855963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=761aa53f47854458b37504bcf29c280b)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:55.858636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:55.871225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:55.872615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:55.883053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 900/1797] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:55.884113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:55.888400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:55.909627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9939ac66-8cc5-4a31-bdb5-339f04b5d693 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:55.919697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ee46e-cacb-4659-8a48-eecedfd91ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:55.920618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee46e-cacb-4659-8a48-eecedfd91ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:55.920902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee46e-cacb-4659-8a48-eecedfd91ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:55.921179 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ee46e-cacb-4659-8a48-eecedfd91ad1 admin admin] PATH_INFO: `/v3/projects/35959159508a439d8fc5b7b9ceb4d69c/users/c19ba3d3722646fd9e391c5b5b3e0f43/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:55.943842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ee46e-cacb-4659-8a48-eecedfd91ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35959159508a439d8fc5b7b9ceb4d69c, user_id=c19ba3d3722646fd9e391c5b5b3e0f43, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:55.946829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ee46e-cacb-4659-8a48-eecedfd91ad1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:56.031170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 901/1798] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:55 2022] PUT /identity/v3/projects/35959159508a439d8fc5b7b9ceb4d69c/users/c19ba3d3722646fd9e391c5b5b3e0f43/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:56.036272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463e5df-44c4-4b9a-8edf-893af341a34c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.036272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463e5df-44c4-4b9a-8edf-893af341a34c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.036471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463e5df-44c4-4b9a-8edf-893af341a34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.067487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2463e5df-44c4-4b9a-8edf-893af341a34c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:56.068707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2463e5df-44c4-4b9a-8edf-893af341a34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:56.136341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 902/1799] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:56.140942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463e5df-44c4-4b9a-8edf-893af341a34c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:56.147624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '761aa53f47854458b37504bcf29c280b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.148416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '761aa53f47854458b37504bcf29c280b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.149600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463e5df-44c4-4b9a-8edf-893af341a34c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:56.154383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:56.155515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.155801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.155966 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '761aa53f47854458b37504bcf29c280b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.156098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.156519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '761aa53f47854458b37504bcf29c280b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.156813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:56.159033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:56.173154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 903/1800] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:56.174510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:56.177737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:56.182327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '761aa53f47854458b37504bcf29c280b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.191622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a0a89e-e00f-4ba7-b148-7b4da1069330 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:56.196111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:56.197127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.197451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.197732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] PATH_INFO: `/v3/users/234d12247c23469aa3710a6aece8317a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.198936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] RBAC: Authorizing `identity:delete_user(user_id=234d12247c23469aa3710a6aece8317a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:56.200895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:56.337888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 898/1801] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:55 2022] DELETE /identity/v3/users/761aa53f47854458b37504bcf29c280b => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:56.338750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:56.344588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:56.445394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01137974-17d0-4e3b-8516-b49948b9a7f6 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:56.450308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:56.453436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.453436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.453436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.453436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:13:56.453436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:56.454403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:56.475165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 899/1802] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:13:56.479954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:56.483008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:56.497159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f50e9-3099-498c-8e1e-5ddf08bbff22 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:56.510932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:56.512661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.513169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.513953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] PATH_INFO: `/v3/projects/35959159508a439d8fc5b7b9ceb4d69c/users/c19ba3d3722646fd9e391c5b5b3e0f43/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.516839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] RBAC: Authorizing `identity:create_grant(project_id=35959159508a439d8fc5b7b9ceb4d69c, user_id=c19ba3d3722646fd9e391c5b5b3e0f43, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:56.520983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:56.524510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234d12247c23469aa3710a6aece8317a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.536744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 900/1803] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:56 2022] PUT /identity/v3/projects/35959159508a439d8fc5b7b9ceb4d69c/users/c19ba3d3722646fd9e391c5b5b3e0f43/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:56.537837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:56.539998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:56.581930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234d12247c23469aa3710a6aece8317a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.581930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234d12247c23469aa3710a6aece8317a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.602669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234d12247c23469aa3710a6aece8317a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.641367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f129bf6-bff9-48d8-b4c1-2d4aa3ff777e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:56.647216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:56.648113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.648614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.649040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] PATH_INFO: `/v3/projects/ee7a9efc0b0d4a61a710ce9d2465c055` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.650295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee7a9efc0b0d4a61a710ce9d2465c055)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:56.652400 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:56.673010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bc4b8de9-9195-47f2-8181-c7dd96a89109 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234d12247c23469aa3710a6aece8317a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.681063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 904/1804] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:56 2022] DELETE /identity/v3/users/234d12247c23469aa3710a6aece8317a => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:56.682687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.682982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.683262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.704758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:56.707688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:56.709522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:56.788951 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 905/1805] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:56.789868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:56.793522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:56.816218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47759813-78c2-43d8-a2f9-32c0a4734dd3 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:56.830028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee7a9efc0b0d4a61a710ce9d2465c055', 'request_id': 'req-f1daf9e9-80a8-4729-b12c-913c12848184'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:56.900252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5bc87-5e81-42a6-a4e0-bc429ad4f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:56.908049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5bc87-5e81-42a6-a4e0-bc429ad4f084 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:56.908049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5bc87-5e81-42a6-a4e0-bc429ad4f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:56.908049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5bc87-5e81-42a6-a4e0-bc429ad4f084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:56.908049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5bc87-5e81-42a6-a4e0-bc429ad4f084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:56.908049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5bc87-5e81-42a6-a4e0-bc429ad4f084 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:57.002438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee7a9efc0b0d4a61a710ce9d2465c055', 'request_id': 'req-f1daf9e9-80a8-4729-b12c-913c12848184'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.009183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee7a9efc0b0d4a61a710ce9d2465c055', 'request_id': 'req-f1daf9e9-80a8-4729-b12c-913c12848184'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.032414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] The token cache is being invalidate because project ee7a9efc0b0d4a61a710ce9d2465c055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:57.032711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.033647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f1daf9e9-80a8-4729-b12c-913c12848184 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.037901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 901/1806] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:56 2022] DELETE /identity/v3/projects/ee7a9efc0b0d4a61a710ce9d2465c055 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:57.056909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 906/1807] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:56 2022] POST /identity/v3/projects => generated 373 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:57.057885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102aeaeb-982f-4d56-b0df-90ee5e90e298 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.058172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102aeaeb-982f-4d56-b0df-90ee5e90e298 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.059204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.059966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102aeaeb-982f-4d56-b0df-90ee5e90e298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.062529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.062831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.098133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-102aeaeb-982f-4d56-b0df-90ee5e90e298 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:57.099701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-102aeaeb-982f-4d56-b0df-90ee5e90e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.117161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] MFA Rules not processed for user `c19ba3d3722646fd9e391c5b5b3e0f43`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:57.118841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.203807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 907/1808] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:57.205880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db894904-56e0-49c8-8d51-9a64c4897c3c None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.206751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db894904-56e0-49c8-8d51-9a64c4897c3c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.207439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db894904-56e0-49c8-8d51-9a64c4897c3c None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.211407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 908/1809] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:13:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:13:57.213029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db894904-56e0-49c8-8d51-9a64c4897c3c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:57.215585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-db894904-56e0-49c8-8d51-9a64c4897c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.222913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 902/1810] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:57.223898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:57.226035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.228522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:57.247000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747f789-091d-44cb-8339-4ad8b9992c10 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:57.256626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:57.258002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.258659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.259166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] PATH_INFO: `/v3/users/56865cfb20f9489dacc6a8b60bac4bb9` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.263548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56865cfb20f9489dacc6a8b60bac4bb9)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:57.265667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:57.317887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db894904-56e0-49c8-8d51-9a64c4897c3c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:57.322292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:57.323075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.323494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.324303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.326819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.340679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:57.342288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:57.354104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 909/1811] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:57.356489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:57.358590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.372766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131232fd-494b-469b-a6fd-3c427b18059f service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:57.378512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:57.379124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.379995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.379995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.380651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:57.382712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:57.551029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 910/1812] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:57 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:57.552641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:57.555512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.567232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56865cfb20f9489dacc6a8b60bac4bb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.568323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56865cfb20f9489dacc6a8b60bac4bb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.577396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56865cfb20f9489dacc6a8b60bac4bb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.578835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56865cfb20f9489dacc6a8b60bac4bb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.586385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc6b60-3a79-4c92-9d50-e6c12e807472 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:57.598614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:57.599727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.600225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.600716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.603742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.618996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6b7b4ee6-92f1-406d-94c5-e356bd340cd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56865cfb20f9489dacc6a8b60bac4bb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:57.639783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:57.641614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:57.674680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 911/1813] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:57.678042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:57.680530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.774782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 903/1814] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:57 2022] DELETE /identity/v3/users/56865cfb20f9489dacc6a8b60bac4bb9 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:57.782319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.782838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.783832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.837659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeeca1e-b487-44c0-8d98-20bca1947b59 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:57.866391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:57.868490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:57.970527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:57.973011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:57.973495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:57.973949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:57.981228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.048406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 904/1815] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:58.049417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.050896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.088713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.090365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.101373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 912/1816] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:58.101965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.103423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.119225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3c9e5-32e4-4d5b-b14f-95ba8ceca0c9 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.130659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.130659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.130659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.130659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.137460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb455c0-aecb-4611-9f93-20124c5644ce None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.140587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.142359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.142579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.148975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.149260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.149556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.150316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.151936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.157140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 913/1817] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:58.183834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.191474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 905/1818] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:58.196862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e740083d-7224-448f-bbd5-76c679c1dd5c service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.203728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.204411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.204692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.204968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.205984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.207843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.222676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 914/1819] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:58.223941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.238879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1046269-5fa0-4c5a-9077-589b7affab15 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.239863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.242546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.246274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9b2ed-0bbe-41d4-8490-9df50dbb40fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.247617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b2ed-0bbe-41d4-8490-9df50dbb40fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.248125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b2ed-0bbe-41d4-8490-9df50dbb40fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.248607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9b2ed-0bbe-41d4-8490-9df50dbb40fe admin admin] PATH_INFO: `/v3/projects/51311aaf82cb4d59a91b882554e8910a/users/e52974da107f4e749d8799351aa76a07/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.257567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c568db90-09b0-4fc5-8f81-6f020c6d54f7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.262453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.268405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.268405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.268405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] PATH_INFO: `/v3/users/9fd4409ec039457c9bd3a1eb3306629e` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.268405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fd4409ec039457c9bd3a1eb3306629e)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.268405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.271039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9b2ed-0bbe-41d4-8490-9df50dbb40fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=51311aaf82cb4d59a91b882554e8910a, user_id=e52974da107f4e749d8799351aa76a07, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.274289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9b2ed-0bbe-41d4-8490-9df50dbb40fe admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.397633 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 906/1820] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:13:58 2022] PUT /identity/v3/projects/51311aaf82cb4d59a91b882554e8910a/users/e52974da107f4e749d8799351aa76a07/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:58.408389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cdc3d-e621-4811-ae00-da31d5ac81ed None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.408829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cdc3d-e621-4811-ae00-da31d5ac81ed None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.409397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433cdc3d-e621-4811-ae00-da31d5ac81ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.439229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-433cdc3d-e621-4811-ae00-da31d5ac81ed None None] MFA Rules not processed for user `e52974da107f4e749d8799351aa76a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:58.441541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-433cdc3d-e621-4811-ae00-da31d5ac81ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.531053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 907/1821] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:58.535992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cdc3d-e621-4811-ae00-da31d5ac81ed None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.587743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fd4409ec039457c9bd3a1eb3306629e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:58.600206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fd4409ec039457c9bd3a1eb3306629e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:58.601594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fd4409ec039457c9bd3a1eb3306629e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:58.620162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433cdc3d-e621-4811-ae00-da31d5ac81ed None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.623719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fd4409ec039457c9bd3a1eb3306629e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:58.629094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c672c8-a559-46ce-a04c-1cd5f576b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.630240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c672c8-a559-46ce-a04c-1cd5f576b048 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.630530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c672c8-a559-46ce-a04c-1cd5f576b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.630822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c672c8-a559-46ce-a04c-1cd5f576b048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.632078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c672c8-a559-46ce-a04c-1cd5f576b048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.634057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c672c8-a559-46ce-a04c-1cd5f576b048 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.781776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fd4409ec039457c9bd3a1eb3306629e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:58.791389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 915/1822] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:58 2022] DELETE /identity/v3/users/9fd4409ec039457c9bd3a1eb3306629e => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:58.793238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.795450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.802918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 908/1823] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:13:58 2022] POST /identity/v3/projects => generated 371 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:58.805024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.806184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.806571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.834176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:13:58.836160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.875813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99cb34-fae2-451b-b2e2-baea14e1f169 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.910113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 909/1824] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:13:58.913231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:58.914650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:58.931156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43e507-d15b-4ca5-a3ae-b41c894ca36d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:58.945735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.947398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.947890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.948318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.949906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.952792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:58.973213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:58.973213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:58.973213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:58.973213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] PATH_INFO: `/v3/projects/29b3588f70224b428ab2976865d0b169` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:58.974751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29b3588f70224b428ab2976865d0b169)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:58.976709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:59.102532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 910/1825] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:13:58 2022] POST /identity/v3/users => generated 292 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:13:59.105301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:59.110801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:59.125720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1c359-36fa-45c3-b4b8-eb7943080201 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:59.132333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:59.133288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:59.133598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:59.134078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] PATH_INFO: `/v3/projects/c378195158134bdab763427401838ace` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:59.135610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c378195158134bdab763427401838ace)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:59.137979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:59.167996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29b3588f70224b428ab2976865d0b169', 'request_id': 'req-116ab3ee-d04c-45af-8621-c1ef9ff24992'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.300655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29b3588f70224b428ab2976865d0b169', 'request_id': 'req-116ab3ee-d04c-45af-8621-c1ef9ff24992'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.317672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29b3588f70224b428ab2976865d0b169', 'request_id': 'req-116ab3ee-d04c-45af-8621-c1ef9ff24992'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.328110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c378195158134bdab763427401838ace', 'request_id': 'req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.351011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] The token cache is being invalidate because project 29b3588f70224b428ab2976865d0b169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:59.351334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.352072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.353780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 916/1826] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:58 2022] DELETE /identity/v3/projects/29b3588f70224b428ab2976865d0b169 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:59.354573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:59.358578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:59.406051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:13:59.460037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ab3ee-d04c-45af-8621-c1ef9ff24992 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:59.469558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c378195158134bdab763427401838ace', 'request_id': 'req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.479739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c378195158134bdab763427401838ace', 'request_id': 'req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.503294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] The token cache is being invalidate because project c378195158134bdab763427401838ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:13:59.503599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.504471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.506636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 911/1827] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:59 2022] DELETE /identity/v3/projects/c378195158134bdab763427401838ace => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:13:59.517655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:59.517655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:59.543521 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:59.544195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:59.546194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:59.546194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:59.548572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:59.636440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d4c008-3b9b-4afb-ae62-16b4508bbd9e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:59.647248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:59.649591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:59.649591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:59.650138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:59.651700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:59.653944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:59.657178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:59.659087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:59.669519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 912/1828] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:13:59.670936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 917/1829] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:13:59.672086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:59.675568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:59.676470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:13:59.682867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:13:59.688389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e66d70-4b29-4857-b893-2a75b912bdf8 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:59.695085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:59.695742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b2d29-042f-4a59-8de4-4049d47ce4dc admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:13:59.696443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:59.696857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:59.697173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] PATH_INFO: `/v3/projects/2255de4951a54bf1bdbdb04946f51014` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:59.698720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2255de4951a54bf1bdbdb04946f51014)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:59.700952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:59.701339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:13:59.705803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:13:59.705803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:13:59.705803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] PATH_INFO: `/v3/users/547309cea5b947148d7eb83abf459db3` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:13:59.705803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=547309cea5b947148d7eb83abf459db3)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:13:59.707970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:13:59.949863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '547309cea5b947148d7eb83abf459db3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.950599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '547309cea5b947148d7eb83abf459db3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.960601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '547309cea5b947148d7eb83abf459db3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.961197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2255de4951a54bf1bdbdb04946f51014', 'request_id': 'req-c76cd631-6df0-4fff-9934-f6720984f763'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.961743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '547309cea5b947148d7eb83abf459db3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.980793 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '547309cea5b947148d7eb83abf459db3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:13:59.999575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:00.055354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2255de4951a54bf1bdbdb04946f51014', 'request_id': 'req-c76cd631-6df0-4fff-9934-f6720984f763'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.062232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2255de4951a54bf1bdbdb04946f51014', 'request_id': 'req-c76cd631-6df0-4fff-9934-f6720984f763'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.073635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] The token cache is being invalidate because project 2255de4951a54bf1bdbdb04946f51014 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:00.074061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.074951 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.076695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 918/1830] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:13:59 2022] DELETE /identity/v3/projects/2255de4951a54bf1bdbdb04946f51014 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:00.077566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:00.081671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:00.158645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 913/1831] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:13:59 2022] DELETE /identity/v3/users/547309cea5b947148d7eb83abf459db3 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:00.164327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:00.173612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:00.173742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76cd631-6df0-4fff-9934-f6720984f763 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:00.266375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57384876-9e99-4b60-ac24-e37b9f8f3dcb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:00.267344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:00.268351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.269030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.269625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] PATH_INFO: `/v3/users/f27eccd6c9e246b297e1b14ee8ebb055` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:00.271152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f27eccd6c9e246b297e1b14ee8ebb055)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:00.272372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d120b-2737-4161-92a9-ec6dfba97ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:00.273573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d120b-2737-4161-92a9-ec6dfba97ec5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.273689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:00.274027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d120b-2737-4161-92a9-ec6dfba97ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.274422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d120b-2737-4161-92a9-ec6dfba97ec5 admin admin] PATH_INFO: `/v3/projects/d287c3fc73684c54a916a9db177600b4/users/482065b23811445586664da54c944772/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:00.294942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d120b-2737-4161-92a9-ec6dfba97ec5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d287c3fc73684c54a916a9db177600b4, user_id=482065b23811445586664da54c944772, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:00.298585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d120b-2737-4161-92a9-ec6dfba97ec5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:00.439801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 914/1832] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:00 2022] PUT /identity/v3/projects/d287c3fc73684c54a916a9db177600b4/users/482065b23811445586664da54c944772/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:00.439801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.439801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.439801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:00.454944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:00.456349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:00.547651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 915/1833] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:00.547651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:00.547651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:00.560537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476af86-f86c-4c42-bde4-b159bc96fb38 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:00.570924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84433a-6067-4cbc-a6ae-d2eeacd84c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:00.571651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84433a-6067-4cbc-a6ae-d2eeacd84c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.572001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84433a-6067-4cbc-a6ae-d2eeacd84c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.572280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84433a-6067-4cbc-a6ae-d2eeacd84c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:00.573058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84433a-6067-4cbc-a6ae-d2eeacd84c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:00.574578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f27eccd6c9e246b297e1b14ee8ebb055'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.574779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84433a-6067-4cbc-a6ae-d2eeacd84c9c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:00.596855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f27eccd6c9e246b297e1b14ee8ebb055'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.601325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 916/1834] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:00.601489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f27eccd6c9e246b297e1b14ee8ebb055'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.601601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af9ce-11ba-411c-badc-72dfba29668c None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.601601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af9ce-11ba-411c-badc-72dfba29668c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.601601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81af9ce-11ba-411c-badc-72dfba29668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:00.640760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f27eccd6c9e246b297e1b14ee8ebb055'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.648423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b81af9ce-11ba-411c-badc-72dfba29668c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:00.649706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b81af9ce-11ba-411c-badc-72dfba29668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:00.671316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 917/1835] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:00.671316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af9ce-11ba-411c-badc-72dfba29668c None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:00.695067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81af9ce-11ba-411c-badc-72dfba29668c None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:00.701314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:00.702512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.702798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.703075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] PATH_INFO: `/v3/projects/d287c3fc73684c54a916a9db177600b4/users/482065b23811445586664da54c944772/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:00.710645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d287c3fc73684c54a916a9db177600b4, user_id=482065b23811445586664da54c944772, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:00.714808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:00.740892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 918/1836] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:00 2022] PUT /identity/v3/projects/d287c3fc73684c54a916a9db177600b4/users/482065b23811445586664da54c944772/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:00.742411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:00.744492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:00.772045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f27eccd6c9e246b297e1b14ee8ebb055'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:00.781088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 919/1837] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:00 2022] DELETE /identity/v3/users/f27eccd6c9e246b297e1b14ee8ebb055 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:00.782058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:00.791837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:00.886235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02bb5e-1b55-4b00-a43f-71103b869179 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:00.928276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9527f-5c7f-4e27-bec6-1be26c9c0b81 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:00.941692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e296fea-8013-44ad-9e0b-2bfe4a430f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:00.943087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e296fea-8013-44ad-9e0b-2bfe4a430f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.943561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e296fea-8013-44ad-9e0b-2bfe4a430f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.943996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e296fea-8013-44ad-9e0b-2bfe4a430f87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:00.944997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e296fea-8013-44ad-9e0b-2bfe4a430f87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:00.945512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e296fea-8013-44ad-9e0b-2bfe4a430f87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:00.947432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e296fea-8013-44ad-9e0b-2bfe4a430f87 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:00.978915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 920/1838] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:00.982745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466f9d-4957-4921-8d1a-e9c1e3997c59 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:00.983732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466f9d-4957-4921-8d1a-e9c1e3997c59 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:00.984849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0466f9d-4957-4921-8d1a-e9c1e3997c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.026015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:01.027875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.028467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.029638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] PATH_INFO: `/v3/projects/a2b7a182f2e94b3a9e1ca2d88ac23206` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.032336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b7a182f2e94b3a9e1ca2d88ac23206)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:01.034913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-d0466f9d-4957-4921-8d1a-e9c1e3997c59 None None] MFA Rules not processed for user `482065b23811445586664da54c944772`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:01.036272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d0466f9d-4957-4921-8d1a-e9c1e3997c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.036627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:01.127615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 921/1839] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:01.128798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0466f9d-4957-4921-8d1a-e9c1e3997c59 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:01.151797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0466f9d-4957-4921-8d1a-e9c1e3997c59 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:01.156839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:01.157551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.157836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.158117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.159039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:01.166256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:01.199008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b7a182f2e94b3a9e1ca2d88ac23206', 'request_id': 'req-612145b6-fec5-4435-acdb-1f86b31a9bc0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.290931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b7a182f2e94b3a9e1ca2d88ac23206', 'request_id': 'req-612145b6-fec5-4435-acdb-1f86b31a9bc0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.300767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b7a182f2e94b3a9e1ca2d88ac23206', 'request_id': 'req-612145b6-fec5-4435-acdb-1f86b31a9bc0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.313431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 922/1840] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:01 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:01.316513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:01.320211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] The token cache is being invalidate because project a2b7a182f2e94b3a9e1ca2d88ac23206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:01.320650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.322241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.323088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.328442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 919/1841] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:00 2022] DELETE /identity/v3/projects/a2b7a182f2e94b3a9e1ca2d88ac23206 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:01.328442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:01.328931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.404043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba5421-41cc-4d58-8f86-e512d3d2110c admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:01.416740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:01.419331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.419634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.419966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.420841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612145b6-fec5-4435-acdb-1f86b31a9bc0 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:01.424566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.530860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:01.546704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:01.550870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.551166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.551450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] PATH_INFO: `/v3/projects/f02061f380ab44f7a9964dc620ec79f5` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.553785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02061f380ab44f7a9964dc620ec79f5)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:01.555771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:01.567028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:01.568629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:01.585244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 923/1842] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:01.585244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:01.585519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.602879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd3a6a-7019-4f24-8ab0-64401b0d8c6e service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:01.609483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:01.612627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.612627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.612627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.613463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:01.615905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:01.738648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 924/1843] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:01 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:01.738648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:01.738648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.753738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31b379-8f27-416e-8032-d94be399040d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:01.759425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02061f380ab44f7a9964dc620ec79f5', 'request_id': 'req-9486521a-8e1e-45a5-8ff7-b0cface6a626'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.806445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:01.848633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:01.848633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.848633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.848633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] PATH_INFO: `/v3/users/c19ba3d3722646fd9e391c5b5b3e0f43` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.849018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c19ba3d3722646fd9e391c5b5b3e0f43)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:01.849096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:01.870837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02061f380ab44f7a9964dc620ec79f5', 'request_id': 'req-9486521a-8e1e-45a5-8ff7-b0cface6a626'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.878447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02061f380ab44f7a9964dc620ec79f5', 'request_id': 'req-9486521a-8e1e-45a5-8ff7-b0cface6a626'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.889881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] The token cache is being invalidate because project f02061f380ab44f7a9964dc620ec79f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:01.890236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.890899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9486521a-8e1e-45a5-8ff7-b0cface6a626 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:01.892506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 920/1844] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:01 2022] DELETE /identity/v3/projects/f02061f380ab44f7a9964dc620ec79f5 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:01.898785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fbf6-a728-41bc-b084-d3a6e6a55d71 None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.899113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fbf6-a728-41bc-b084-d3a6e6a55d71 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.899422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b603fbf6-a728-41bc-b084-d3a6e6a55d71 None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.900767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 921/1845] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:14:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:14:01.901538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603fbf6-a728-41bc-b084-d3a6e6a55d71 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:01.903212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b603fbf6-a728-41bc-b084-d3a6e6a55d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.918853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b603fbf6-a728-41bc-b084-d3a6e6a55d71 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:01.923871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012762a1-695b-4f15-9d8b-fdd9222d6a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:01.924710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012762a1-695b-4f15-9d8b-fdd9222d6a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.924995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012762a1-695b-4f15-9d8b-fdd9222d6a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.925308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012762a1-695b-4f15-9d8b-fdd9222d6a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.926068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012762a1-695b-4f15-9d8b-fdd9222d6a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:01.927705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012762a1-695b-4f15-9d8b-fdd9222d6a30 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:01.942606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 922/1846] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:01.948957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e2f1f-2765-4459-a087-587337b45a2a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.949384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e2f1f-2765-4459-a087-587337b45a2a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.949673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e2f1f-2765-4459-a087-587337b45a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:01.969995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-576e2f1f-2765-4459-a087-587337b45a2a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:01.972121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-576e2f1f-2765-4459-a087-587337b45a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:01.990586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 923/1847] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:01.992172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca9b3d-bc40-4ed1-b0ca-4c3e40995e5a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:01.992461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca9b3d-bc40-4ed1-b0ca-4c3e40995e5a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:01.992742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca9b3d-bc40-4ed1-b0ca-4c3e40995e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.013956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e5ca9b3d-bc40-4ed1-b0ca-4c3e40995e5a None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:02.015148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e5ca9b3d-bc40-4ed1-b0ca-4c3e40995e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.076683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c19ba3d3722646fd9e391c5b5b3e0f43'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.089731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c19ba3d3722646fd9e391c5b5b3e0f43'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.090589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c19ba3d3722646fd9e391c5b5b3e0f43'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.103951 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c19ba3d3722646fd9e391c5b5b3e0f43'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.115044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 924/1848] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:14:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:02.115911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca9b3d-bc40-4ed1-b0ca-4c3e40995e5a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.126904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca9b3d-bc40-4ed1-b0ca-4c3e40995e5a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:02.131809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:02.132484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.133008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.133317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.134411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:02.135644 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:02.147283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 925/1849] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:02.148102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.150292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.170293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d118c1-6ccb-41f6-82fb-d28b8397b878 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:02.175739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:02.176593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.176875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.177149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] PATH_INFO: `/v3/projects/c251bfe6e28f47cba472557fca2e2c2d` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.178345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c251bfe6e28f47cba472557fca2e2c2d)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:02.180294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:02.244397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d02a476d-ef47-4260-b1d9-5aadae850332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c19ba3d3722646fd9e391c5b5b3e0f43'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.250840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 925/1850] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:01 2022] DELETE /identity/v3/users/c19ba3d3722646fd9e391c5b5b3e0f43 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:02.252112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.252393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.252744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.272540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:02.273193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.331184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c251bfe6e28f47cba472557fca2e2c2d', 'request_id': 'req-d1d38774-4a64-4344-994c-ecdb8b6863fd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.338201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 926/1851] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:02.339063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.340391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.430663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c251bfe6e28f47cba472557fca2e2c2d', 'request_id': 'req-d1d38774-4a64-4344-994c-ecdb8b6863fd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.435557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f183a82-664d-41fe-8ba8-7ccd4c9827d0 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:02.437786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c251bfe6e28f47cba472557fca2e2c2d', 'request_id': 'req-d1d38774-4a64-4344-994c-ecdb8b6863fd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.456983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] The token cache is being invalidate because project c251bfe6e28f47cba472557fca2e2c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:02.457283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.466178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:02.474139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 926/1852] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:02 2022] DELETE /identity/v3/projects/c251bfe6e28f47cba472557fca2e2c2d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:02.475881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.479668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.546563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:02.547780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.548169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.548553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.552049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.567888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:02.569711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:02.583893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 927/1853] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:02.584082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d38774-4a64-4344-994c-ecdb8b6863fd admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:02.591804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:02.592929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.593382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.593897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] PATH_INFO: `/v3/projects/a945d8008e5340889485c0a852485ba4/users/b84822af3776408995fa4cc6c9b39b01/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.595470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.601035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.617172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a945d8008e5340889485c0a852485ba4, user_id=b84822af3776408995fa4cc6c9b39b01, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:02.621360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:02.632980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c408b-7c18-4986-a8fa-278a0bd5471b service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:02.640030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83ec3b-5e39-4bf7-971e-b8c52cfb1fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:02.641024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83ec3b-5e39-4bf7-971e-b8c52cfb1fee admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.641440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83ec3b-5e39-4bf7-971e-b8c52cfb1fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.641837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83ec3b-5e39-4bf7-971e-b8c52cfb1fee admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.643361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83ec3b-5e39-4bf7-971e-b8c52cfb1fee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:02.643361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83ec3b-5e39-4bf7-971e-b8c52cfb1fee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:02.645746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf83ec3b-5e39-4bf7-971e-b8c52cfb1fee admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:02.687808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 928/1854] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:02.690097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494a2f-12f5-405e-80b1-008a8e3e5ddd None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.690505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494a2f-12f5-405e-80b1-008a8e3e5ddd None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.690665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb494a2f-12f5-405e-80b1-008a8e3e5ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.714151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 927/1855] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:02 2022] PUT /identity/v3/projects/a945d8008e5340889485c0a852485ba4/users/b84822af3776408995fa4cc6c9b39b01/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:02.725833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.727703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fb494a2f-12f5-405e-80b1-008a8e3e5ddd None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:02.729092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fb494a2f-12f5-405e-80b1-008a8e3e5ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.818078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b06b6-bb33-4c10-b407-21d9dc3c7073 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:02.831442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:02.832555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.832714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.833001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.833984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:02.835718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:02.847766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 929/1856] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:02.853336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:02.853651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:02.853940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:02.890112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] MFA Rules not processed for user `b84822af3776408995fa4cc6c9b39b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:02.891327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.957390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 930/1857] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:02.958355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.959712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.968706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 928/1858] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:02 2022] POST /identity/v3/projects => generated 379 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:02.970841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:02.973459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:02.975479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165f7450-eceb-4f0d-90df-bd923038b116 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.055392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.056292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.056580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.056859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] PATH_INFO: `/v3/projects/35959159508a439d8fc5b7b9ceb4d69c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.058338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35959159508a439d8fc5b7b9ceb4d69c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.060321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201c284-20a8-41ca-9961-d79f72a13042 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.061025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.067150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.068604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.068604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.069050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] PATH_INFO: `/v3/users/e52974da107f4e749d8799351aa76a07` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.070800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e52974da107f4e749d8799351aa76a07)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.073204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.303860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35959159508a439d8fc5b7b9ceb4d69c', 'request_id': 'req-f0b6b740-606e-4bed-b706-3a83efeecaa2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.308192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e52974da107f4e749d8799351aa76a07'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.309662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e52974da107f4e749d8799351aa76a07'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.321568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e52974da107f4e749d8799351aa76a07'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.323416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e52974da107f4e749d8799351aa76a07'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.340468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e52974da107f4e749d8799351aa76a07'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.443819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35959159508a439d8fc5b7b9ceb4d69c', 'request_id': 'req-f0b6b740-606e-4bed-b706-3a83efeecaa2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.451013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35959159508a439d8fc5b7b9ceb4d69c', 'request_id': 'req-f0b6b740-606e-4bed-b706-3a83efeecaa2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.462210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] The token cache is being invalidate because project 35959159508a439d8fc5b7b9ceb4d69c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:03.462507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.463159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0b6b740-606e-4bed-b706-3a83efeecaa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:03.465013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 931/1859] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:02 2022] DELETE /identity/v3/projects/35959159508a439d8fc5b7b9ceb4d69c => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:03.466096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.466386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.466668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.485787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:03.487496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:03.548239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 932/1860] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:03.549669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:03.550943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:03.562127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf2f927-c9de-4393-9434-6d37ef141c7d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.566984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955ecd73-f964-4290-8a52-c4a5524b327d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.567622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ecd73-f964-4290-8a52-c4a5524b327d admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.570863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ecd73-f964-4290-8a52-c4a5524b327d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.570863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955ecd73-f964-4290-8a52-c4a5524b327d admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.570863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ecd73-f964-4290-8a52-c4a5524b327d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.570863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955ecd73-f964-4290-8a52-c4a5524b327d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.598917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 929/1861] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:02 2022] DELETE /identity/v3/users/e52974da107f4e749d8799351aa76a07 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:03.599830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:03.601422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:03.665835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec5a60-3b8e-4645-8beb-01f8982d4c35 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.670681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3929d5-fd02-4aa6-ba6b-c315f1edd584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.673372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3929d5-fd02-4aa6-ba6b-c315f1edd584 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.673776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3929d5-fd02-4aa6-ba6b-c315f1edd584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.674183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3929d5-fd02-4aa6-ba6b-c315f1edd584 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.675055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3929d5-fd02-4aa6-ba6b-c315f1edd584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.677040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3929d5-fd02-4aa6-ba6b-c315f1edd584 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.683069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 933/1862] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/projects => generated 361 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:03.683069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841674a-6f28-47f9-a316-d47608ea5d94 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.683069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841674a-6f28-47f9-a316-d47608ea5d94 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.683069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b841674a-6f28-47f9-a316-d47608ea5d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.701733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b841674a-6f28-47f9-a316-d47608ea5d94 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:03.702968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b841674a-6f28-47f9-a316-d47608ea5d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:03.770194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 934/1863] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:03.771189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841674a-6f28-47f9-a316-d47608ea5d94 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:03.780640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b841674a-6f28-47f9-a316-d47608ea5d94 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.786818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.787574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.787862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.788139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.788880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:03.789192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.790824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.802188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 930/1864] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:03.804603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4075c32-cba8-41fa-8a81-b49afa6f5467 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.805000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4075c32-cba8-41fa-8a81-b49afa6f5467 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.805407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4075c32-cba8-41fa-8a81-b49afa6f5467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.809360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 935/1865] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:03.812212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:03.823352 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0e98c-3941-44cc-ab7c-a214272d5e98 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.825201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:03.827707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b4075c32-cba8-41fa-8a81-b49afa6f5467 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:03.828903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b4075c32-cba8-41fa-8a81-b49afa6f5467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:03.829369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.831054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.831554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.832041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.833039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.835761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.838632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 931/1866] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:03.840265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.840627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.840973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.860682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:03.862256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:03.870661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 932/1867] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:03.871519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:03.874433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:03.886317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a146c3-a93b-423a-8f19-d3339e4438f3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.890915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042774d1-3b05-4747-a212-110559585892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.891576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042774d1-3b05-4747-a212-110559585892 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.891859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042774d1-3b05-4747-a212-110559585892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.892138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042774d1-3b05-4747-a212-110559585892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.893441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042774d1-3b05-4747-a212-110559585892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.895121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042774d1-3b05-4747-a212-110559585892 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.905559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 933/1868] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:03.906599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042774d1-3b05-4747-a212-110559585892 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:03.916990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042774d1-3b05-4747-a212-110559585892 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:03.921724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:03.922672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:03.922955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:03.923237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:03.924557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:03.926504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:03.961167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 936/1869] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/users => generated 287 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:03.962060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:03.964407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:04.021657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b8e32-8112-41c3-954d-c44560719eef admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.025640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.026324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.026608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.026908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 934/1870] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:03 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:04.027017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.027704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.036878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.038372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.048524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 937/1871] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:04.049760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.062837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e42238-6d42-4b4b-bd67-e7f871a96243 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.105193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25345591-8656-4c3b-bb9a-849e63be97b7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.110359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.111023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.111498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.111732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.112645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:04.112960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.120181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.130331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 935/1872] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:04.131292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.138683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.139584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.140461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.140874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.144020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.146074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.146887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8e2fe-b996-4e0d-8adf-047d9bbeb8ee admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.151914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.152641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.152927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.153203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.154151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.155892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.163856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 938/1873] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:04.165079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.169244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 936/1874] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:04.170568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.177353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bac32b-ef58-4277-a120-79ef3001a09f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.183435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2958f601-70e4-4cb4-96d7-d7df328fdf4b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.187156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.187852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.188136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.188415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.189166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.191682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.192968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.193973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.194530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.195173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.196121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.199867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.390757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 937/1875] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:04 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:04.391931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.395996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 939/1876] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:04 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:04.396899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.417231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1510431e-a7ee-41c5-a58d-ff437265a488 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.484246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38085-8fc6-499f-973c-30b40ef8dc5f admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.491569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.493543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.493543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.493543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] PATH_INFO: `/v3/projects/01cf7d8d9d06491d8eda6230055967da/users/cc60dc79dee140118712b9f7e4d58fba/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.508182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.509081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.509398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.509735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] PATH_INFO: `/v3/projects/d39b0ab1d56141f0ba2c54a3ae528373/users/7d0dc56d450c43aebd29afc11613331e/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.510093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01cf7d8d9d06491d8eda6230055967da, user_id=cc60dc79dee140118712b9f7e4d58fba, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.512842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.526453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39b0ab1d56141f0ba2c54a3ae528373, user_id=7d0dc56d450c43aebd29afc11613331e, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.529899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.607993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 938/1877] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:04 2022] PUT /identity/v3/projects/01cf7d8d9d06491d8eda6230055967da/users/cc60dc79dee140118712b9f7e4d58fba/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:04.607993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.614432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 940/1878] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:04 2022] PUT /identity/v3/projects/d39b0ab1d56141f0ba2c54a3ae528373/users/7d0dc56d450c43aebd29afc11613331e/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:04.614914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.659952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:04.679262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12947c0-84bc-4371-ae83-e0769a8db55f admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.684553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.685224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.686425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.686754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.690288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.690560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.694108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b873c684-083b-41ab-b546-5d749e03441b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.698811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f8d36-37c0-4b4b-9fc3-4a86e997f882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.699490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f8d36-37c0-4b4b-9fc3-4a86e997f882 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.699775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f8d36-37c0-4b4b-9fc3-4a86e997f882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.700052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f8d36-37c0-4b4b-9fc3-4a86e997f882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.700805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f8d36-37c0-4b4b-9fc3-4a86e997f882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.702435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f8d36-37c0-4b4b-9fc3-4a86e997f882 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.712478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 941/1879] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:04.713710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.714000 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.714279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.740078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] MFA Rules not processed for user `cc60dc79dee140118712b9f7e4d58fba`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:04.741351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:04.777464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 939/1880] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:04 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:04.778854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.791330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4515968-45ab-4d8a-808b-c55e1a36ac2a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.796398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.797360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.797648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.797954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.799375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.801065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.811305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 940/1881] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:04.813255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.816109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 942/1882] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:04.817502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.818979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:04.824002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c360de-8c0a-4f82-afb4-2a6e589af304 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.828848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.829550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.829825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.830127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.831200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.832742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.833247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa02d9-c3c5-48eb-8fa8-244f626d2d78 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.837953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.838815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.839226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.839504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] PATH_INFO: `/v3/projects/51311aaf82cb4d59a91b882554e8910a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.840680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51311aaf82cb4d59a91b882554e8910a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.842651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.843324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 941/1883] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:04.844227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.854679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ad46f-5ae3-4d8e-95af-0bc04b218411 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.859445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.860101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.860385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.860663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.862170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.863820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.873975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 942/1884] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:04.874862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.884523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61ef3d-2b00-4956-a89e-a7965a5d6600 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:04.889115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:04.890004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:04.890281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:04.890557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] PATH_INFO: `/v3/projects/d39b0ab1d56141f0ba2c54a3ae528373/users/7d0dc56d450c43aebd29afc11613331e/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:04.892466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d39b0ab1d56141f0ba2c54a3ae528373, user_id=7d0dc56d450c43aebd29afc11613331e, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:04.895139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:04.905161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 943/1885] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:04 2022] PUT /identity/v3/projects/d39b0ab1d56141f0ba2c54a3ae528373/users/7d0dc56d450c43aebd29afc11613331e/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:04.905985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:04.941503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51311aaf82cb4d59a91b882554e8910a', 'request_id': 'req-7cc831f6-8848-4005-b166-4466daaf7431'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:04.969533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2d4d6-e421-4c02-9a3f-1615ea964ca3 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.004237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51311aaf82cb4d59a91b882554e8910a', 'request_id': 'req-7cc831f6-8848-4005-b166-4466daaf7431'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.011037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51311aaf82cb4d59a91b882554e8910a', 'request_id': 'req-7cc831f6-8848-4005-b166-4466daaf7431'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.022032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] The token cache is being invalidate because project 51311aaf82cb4d59a91b882554e8910a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:05.022324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.022996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.024904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 943/1886] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:04 2022] DELETE /identity/v3/projects/51311aaf82cb4d59a91b882554e8910a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:05.026494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.028460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.030062 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.030762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.031028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.031299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.032001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.034878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.044423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc831f6-8848-4005-b166-4466daaf7431 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.056224 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfab253-9470-48ed-8b92-bd4ad9024159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.062103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfab253-9470-48ed-8b92-bd4ad9024159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.062103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfab253-9470-48ed-8b92-bd4ad9024159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.062103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfab253-9470-48ed-8b92-bd4ad9024159 admin admin] PATH_INFO: `/v3/projects/b821b9f24df0462ca04377c94d19a170/users/ecfc5d6661004641b1a8c689cd960946/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.077227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfab253-9470-48ed-8b92-bd4ad9024159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b821b9f24df0462ca04377c94d19a170, user_id=ecfc5d6661004641b1a8c689cd960946, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.080087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfab253-9470-48ed-8b92-bd4ad9024159 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.152340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 944/1887] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:04 2022] POST /identity/v3/projects => generated 381 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:05.152340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.152340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.174610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 944/1888] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:05 2022] PUT /identity/v3/projects/b821b9f24df0462ca04377c94d19a170/users/ecfc5d6661004641b1a8c689cd960946/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:05.177382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.182789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.182789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.208625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] MFA Rules not processed for user `7d0dc56d450c43aebd29afc11613331e`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:05.209973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.238593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a4da68-062a-459d-a582-2d9a006d35b5 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.282024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 945/1889] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:05.283135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.284521 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.302045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.303073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.308710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.308710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.308710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.308710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.319455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 945/1890] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:05.320572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.322631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.356781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1d286-15ed-4836-b17f-d78bb87a2cd2 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.370493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece1dd3f-e827-4196-8f69-def58c366b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.371261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1dd3f-e827-4196-8f69-def58c366b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.371604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1dd3f-e827-4196-8f69-def58c366b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.371829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece1dd3f-e827-4196-8f69-def58c366b5f admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.372538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1dd3f-e827-4196-8f69-def58c366b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.374499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece1dd3f-e827-4196-8f69-def58c366b5f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.404669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6dcdc8-d665-418d-b5e6-4e08d5a4bb3f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.410266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:05.410636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.411639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.412062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.412593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] PATH_INFO: `/v3/users/482065b23811445586664da54c944772` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.414667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=482065b23811445586664da54c944772)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.416798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.498997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 946/1891] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:05 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:05.500317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.500687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.500947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.527620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] MFA Rules not processed for user `ecfc5d6661004641b1a8c689cd960946`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:05.528828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.593759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 947/1892] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:05.594830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.596811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.661230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '482065b23811445586664da54c944772'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.664647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082e5a6-42ec-4a58-abe9-6ff30fbd7b6e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.668726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.669563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.669851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.670113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.670520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '482065b23811445586664da54c944772'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.671402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '482065b23811445586664da54c944772'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.681195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.682980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.689602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '482065b23811445586664da54c944772'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.694652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 948/1893] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:05 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:05.702431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.717082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2896955-e384-436d-b194-c1b2b44f90da service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.723854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.724965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.725142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.725475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] PATH_INFO: `/v3/projects/4d47f8efcc7a438c90f33178422cc6db/users/2241e7dcf7b44037b1b80e0f99c8e5e8/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.742169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d47f8efcc7a438c90f33178422cc6db, user_id=2241e7dcf7b44037b1b80e0f99c8e5e8, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.747559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.767740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '482065b23811445586664da54c944772'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:05.774336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 946/1894] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:05 2022] DELETE /identity/v3/users/482065b23811445586664da54c944772 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:05.775221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.776624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.858864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c41c568-06bd-48fa-b10a-1fa48c20f611 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.870254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa3f08-5b45-4856-93ac-adbfcb1a8163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.871102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3f08-5b45-4856-93ac-adbfcb1a8163 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.871566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3f08-5b45-4856-93ac-adbfcb1a8163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.871921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3f08-5b45-4856-93ac-adbfcb1a8163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.873593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa3f08-5b45-4856-93ac-adbfcb1a8163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.874774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 949/1895] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:05 2022] PUT /identity/v3/projects/4d47f8efcc7a438c90f33178422cc6db/users/2241e7dcf7b44037b1b80e0f99c8e5e8/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:05.875544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa3f08-5b45-4856-93ac-adbfcb1a8163 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.875660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:05.880473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.894296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 947/1896] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:05.894296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49765b3d-b927-4394-a781-7c6e23ada810 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.894296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49765b3d-b927-4394-a781-7c6e23ada810 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.894296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49765b3d-b927-4394-a781-7c6e23ada810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.920358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-49765b3d-b927-4394-a781-7c6e23ada810 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:05.922973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-49765b3d-b927-4394-a781-7c6e23ada810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:05.966007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c900a2d-44ad-4bd1-8631-c084467c2d2e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:05.979597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:05.980286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:05.980574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:05.980853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:05.981598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:05.983674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:05.998879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 948/1897] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:05.999870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49765b3d-b927-4394-a781-7c6e23ada810 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.012809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49765b3d-b927-4394-a781-7c6e23ada810 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.018577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.019808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.020009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.020283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] PATH_INFO: `/v3/projects/1d6b1f3afb864b79aea1b44f72588ddf/users/ea07e332a4c3440ea7e4083465192a07/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.036900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6b1f3afb864b79aea1b44f72588ddf, user_id=ea07e332a4c3440ea7e4083465192a07, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.041087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.132861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 949/1898] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:05 2022] PUT /identity/v3/projects/1d6b1f3afb864b79aea1b44f72588ddf/users/ea07e332a4c3440ea7e4083465192a07/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:06.134984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 950/1899] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:05 2022] POST /identity/v3/projects => generated 383 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:06.135306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.141871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:06.144925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.223863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e9397-c04b-4aef-b18b-cb55e4b54eac admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.260567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599c39f-01b2-4606-8988-2e0904dd6901 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.266339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.268645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.269016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.269352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.270113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.272312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.289879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 951/1900] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:06.290579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.290782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.291990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.292457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.292938 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.302740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a8768e-eda0-4fef-a71f-e5036ece3fb3 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.306839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.315081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.315786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.315896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.316173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.316448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.317145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.319105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.335541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 950/1901] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:06.337169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.349978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303f989-7035-4f38-a967-ef6371c990a0 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.356405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c31681-3704-4376-991c-a8132bea2bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.357753 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c31681-3704-4376-991c-a8132bea2bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.365854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c31681-3704-4376-991c-a8132bea2bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.365854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c31681-3704-4376-991c-a8132bea2bd3 admin admin] PATH_INFO: `/v3/projects/1d6b1f3afb864b79aea1b44f72588ddf/users/ea07e332a4c3440ea7e4083465192a07/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.365854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c31681-3704-4376-991c-a8132bea2bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d6b1f3afb864b79aea1b44f72588ddf, user_id=ea07e332a4c3440ea7e4083465192a07, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.365854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c31681-3704-4376-991c-a8132bea2bd3 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.384692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 951/1902] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:06 2022] PUT /identity/v3/projects/1d6b1f3afb864b79aea1b44f72588ddf/users/ea07e332a4c3440ea7e4083465192a07/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:06.393654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc5414-f66f-4eb1-b208-f6d7da20b05e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.397127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc5414-f66f-4eb1-b208-f6d7da20b05e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.397127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc5414-f66f-4eb1-b208-f6d7da20b05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.426755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 952/1903] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:06 2022] POST /identity/v3/users => generated 298 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:06.430938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.432652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-58bc5414-f66f-4eb1-b208-f6d7da20b05e None None] MFA Rules not processed for user `ea07e332a4c3440ea7e4083465192a07`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:06.433923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-58bc5414-f66f-4eb1-b208-f6d7da20b05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:06.443765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838604c-91d6-4e25-9e83-a3e8b98c5708 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.544222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 952/1904] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:06.558193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.558956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.559309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.559866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.560683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.562944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.573846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 953/1905] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:06.578311 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc5414-f66f-4eb1-b208-f6d7da20b05e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.589546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.590851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc5414-f66f-4eb1-b208-f6d7da20b05e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.594432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:06.597757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.603071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.603071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.603071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] PATH_INFO: `/v3/projects/d060b3c8349c4ef184bf8dad91bbc8a6/users/365f283179494b9d822e7befd4a677c9/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.622361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d060b3c8349c4ef184bf8dad91bbc8a6, user_id=365f283179494b9d822e7befd4a677c9, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.627509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.691513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb7a1e-542d-45b1-bdda-fe84e99ab94a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.697573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.698257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.698594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.698879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.721557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.723504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.735848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 954/1906] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:06.736792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.738414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:06.757014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6842c6-9849-4c1d-8837-e858cbbd86e6 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.763543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:06.764421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:06.764709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:06.764991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] PATH_INFO: `/v3/projects/d287c3fc73684c54a916a9db177600b4` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:06.766806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d287c3fc73684c54a916a9db177600b4)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:06.769603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:06.864512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 953/1907] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:06 2022] PUT /identity/v3/projects/d060b3c8349c4ef184bf8dad91bbc8a6/users/365f283179494b9d822e7befd4a677c9/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:06.868338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:06.964423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a17e0-fe40-424b-a3e4-bbd9f4d5f38f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:06.975020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd287c3fc73684c54a916a9db177600b4', 'request_id': 'req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:07.057856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.058746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.059110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.059396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.060165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.062043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.074385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 954/1908] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:07.077950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd287c3fc73684c54a916a9db177600b4', 'request_id': 'req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:07.082150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.088721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd287c3fc73684c54a916a9db177600b4', 'request_id': 'req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:07.093845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701443f4-6a7c-40f8-93cf-cfbe4e8f730a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.099522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aceef63-f979-48f7-b58a-5a660a393378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.100540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceef63-f979-48f7-b58a-5a660a393378 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.100759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceef63-f979-48f7-b58a-5a660a393378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.101001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aceef63-f979-48f7-b58a-5a660a393378 admin admin] PATH_INFO: `/v3/projects/d060b3c8349c4ef184bf8dad91bbc8a6/users/365f283179494b9d822e7befd4a677c9/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.103063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aceef63-f979-48f7-b58a-5a660a393378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d060b3c8349c4ef184bf8dad91bbc8a6, user_id=365f283179494b9d822e7befd4a677c9, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.105229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] The token cache is being invalidate because project d287c3fc73684c54a916a9db177600b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:07.105657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:07.105862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aceef63-f979-48f7-b58a-5a660a393378 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.106750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ee5b81ef-26eb-4820-b1a9-bf466e9ef084 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:07.110642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 955/1909] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:06 2022] DELETE /identity/v3/projects/d287c3fc73684c54a916a9db177600b4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:07.115569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b641300-8b1e-465e-b53e-afbcf8ec9be6 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.115972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b641300-8b1e-465e-b53e-afbcf8ec9be6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.116387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b641300-8b1e-465e-b53e-afbcf8ec9be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.128597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 955/1910] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:07 2022] PUT /identity/v3/projects/d060b3c8349c4ef184bf8dad91bbc8a6/users/365f283179494b9d822e7befd4a677c9/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:07.143591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8951-654a-4785-a7f8-68edc7dc5566 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.143591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8951-654a-4785-a7f8-68edc7dc5566 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.143591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd8951-654a-4785-a7f8-68edc7dc5566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.149848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-6b641300-8b1e-465e-b53e-afbcf8ec9be6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:07.151317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6b641300-8b1e-465e-b53e-afbcf8ec9be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:07.199898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7bbd8951-654a-4785-a7f8-68edc7dc5566 None None] MFA Rules not processed for user `365f283179494b9d822e7befd4a677c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:07.205349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7bbd8951-654a-4785-a7f8-68edc7dc5566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:07.253131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 956/1911] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:07.260529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70a89-5ad7-4bae-b8d1-1fc2e91190d7 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.260824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70a89-5ad7-4bae-b8d1-1fc2e91190d7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.261108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70a89-5ad7-4bae-b8d1-1fc2e91190d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.286744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-bbb70a89-5ad7-4bae-b8d1-1fc2e91190d7 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:07.293547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bbb70a89-5ad7-4bae-b8d1-1fc2e91190d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:07.310417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 957/1912] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:07.315414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb70a89-5ad7-4bae-b8d1-1fc2e91190d7 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.324994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb70a89-5ad7-4bae-b8d1-1fc2e91190d7 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.349153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.350453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.351068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.351368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.352146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:07.353379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.355055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.358009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 956/1913] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:07.379525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 958/1914] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:14:07.385309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8951-654a-4785-a7f8-68edc7dc5566 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.402465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd8951-654a-4785-a7f8-68edc7dc5566 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.411474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.419728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.419728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.419728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.419728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.419728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.437375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.439495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:07.514243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d876eb5-5a95-4a26-9150-5c617062427f admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.519358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.520175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.520536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.520913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.522963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:07.592024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 957/1915] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:07 2022] POST /identity/v3/projects => generated 409 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:07.593245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.598589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.600143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.610384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 959/1916] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:07.707137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3477a3-a1b7-4ef9-8cf2-13b9a5b82276 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.712146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.712858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.713182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.713527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.714241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.716187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.827187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.833003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 958/1917] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:07 2022] POST /identity/v3/users => generated 305 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:07.833864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:07.836025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.848488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f819f-67d4-499e-ace4-58ebb1c270c6 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.856525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.857360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.857682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.858002 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.858566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ccf311-0a3e-44b6-b07c-a75f3512d886 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.858741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.860457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.876542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.881147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.884613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.884613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] PATH_INFO: `/v3/users/b84822af3776408995fa4cc6c9b39b01` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.888664 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 959/1918] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:07.888810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b84822af3776408995fa4cc6c9b39b01)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.890297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:07.895480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.936869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:07.949440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b75a5-d3b3-4b3a-8fd3-6934eb87b8b4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:07.971824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:07.977461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:07.977461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:07.977461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:07.977461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:07.977461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:07.992031 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 960/1919] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:07.996137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.015541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006738aa-5a02-4fe9-938e-21e8b40cef5a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:08.021841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910c57b-15e0-40b8-9de8-d47ac8711fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:08.022984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910c57b-15e0-40b8-9de8-d47ac8711fe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.023355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910c57b-15e0-40b8-9de8-d47ac8711fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.024360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910c57b-15e0-40b8-9de8-d47ac8711fe0 admin admin] PATH_INFO: `/v3/projects/171b48aa7e3341bf8a9ea1b4e256ea4e/users/d60f42ba5bbf4abcaf442cbc24b0a89e/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.045086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910c57b-15e0-40b8-9de8-d47ac8711fe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=171b48aa7e3341bf8a9ea1b4e256ea4e, user_id=d60f42ba5bbf4abcaf442cbc24b0a89e, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:08.048150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910c57b-15e0-40b8-9de8-d47ac8711fe0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:08.123252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 961/1920] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:07 2022] PUT /identity/v3/projects/171b48aa7e3341bf8a9ea1b4e256ea4e/users/d60f42ba5bbf4abcaf442cbc24b0a89e/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:08.134841 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.135128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.135811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.168931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] MFA Rules not processed for user `d60f42ba5bbf4abcaf442cbc24b0a89e`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:08.170723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:08.234946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b84822af3776408995fa4cc6c9b39b01'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:08.237126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b84822af3776408995fa4cc6c9b39b01'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:08.248564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b84822af3776408995fa4cc6c9b39b01'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:08.249306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b84822af3776408995fa4cc6c9b39b01'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:08.264386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b84822af3776408995fa4cc6c9b39b01'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:08.276777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 962/1921] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:08.278277 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.279844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:08.378620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578c6f0-5421-4c6c-9131-c3eb95ecbf15 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:08.385978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0d057-045d-4011-b245-b1ee379341e8 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea07e332a4c3440ea7e4083465192a07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6b1f3afb864b79aea1b44f72588ddf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:08.387352 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d057-045d-4011-b245-b1ee379341e8 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.387734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d057-045d-4011-b245-b1ee379341e8 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.388112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0d057-045d-4011-b245-b1ee379341e8 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.388876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0d057-045d-4011-b245-b1ee379341e8 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:08.389193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0d057-045d-4011-b245-b1ee379341e8 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:08.390956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0d057-045d-4011-b245-b1ee379341e8 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:08.407286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 960/1922] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:07 2022] DELETE /identity/v3/users/b84822af3776408995fa4cc6c9b39b01 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:08.407286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.408656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 963/1923] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:08.409887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26343121-da0a-4d44-8c40-9d264c76d8ce None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.410099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:08.410210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26343121-da0a-4d44-8c40-9d264c76d8ce None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.410447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26343121-da0a-4d44-8c40-9d264c76d8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.459464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-26343121-da0a-4d44-8c40-9d264c76d8ce None None] MFA Rules not processed for user `2241e7dcf7b44037b1b80e0f99c8e5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:08.460843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-26343121-da0a-4d44-8c40-9d264c76d8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:08.511242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c127f-01e6-49fa-beda-7565fd539834 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:08.516810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:08.518241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.518580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.518862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.519563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:08.521495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:08.535008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 964/1924] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:08.536504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d563ba8-2b2c-4088-9f51-5ca936a9cabf None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.536905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d563ba8-2b2c-4088-9f51-5ca936a9cabf None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.537327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d563ba8-2b2c-4088-9f51-5ca936a9cabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.557428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0d563ba8-2b2c-4088-9f51-5ca936a9cabf None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:08.558721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0d563ba8-2b2c-4088-9f51-5ca936a9cabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:08.567056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 965/1925] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:08.567971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d563ba8-2b2c-4088-9f51-5ca936a9cabf None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.632747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d563ba8-2b2c-4088-9f51-5ca936a9cabf None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:08.638116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea07e332a4c3440ea7e4083465192a07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6b1f3afb864b79aea1b44f72588ddf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:08.638796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.639082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.639366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.640197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:08.642154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:08.655198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 961/1926] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:08 2022] POST /identity/v3/projects => generated 403 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:08.656310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.658992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:08.734247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c7a5d-5702-4fae-a32d-182d206a6399 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:08.739173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:08.739940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.745660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.745660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.745660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:08.745660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:08.771597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 966/1927] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:08 2022] POST /identity/v3/projects => generated 354 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:08.774177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.774297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:08.829110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760886e-9a6a-471a-be22-20870abd001d tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:08.833981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:08.834742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.835087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.835360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.842989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 962/1928] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:08 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:08.842989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.904343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:08.905949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:08.909439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:08.914497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f6b29-5283-41df-8089-e58e305c5c36 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:08.916175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 967/1929] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:08.917008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:08.920148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:08.920978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:08.921403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:08.922041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:08.922850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:08.926086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:08.991191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ab890-ee78-497e-a3f5-8fc9d733d793 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.023169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 963/1930] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:08 2022] POST /identity/v3/users => generated 308 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:09.024105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.035217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acaa7f-de2b-41fd-a62a-7297c3797b4e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.040530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.041658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.042047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.042488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.043675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.045312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.056984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea07e332a4c3440ea7e4083465192a07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6b1f3afb864b79aea1b44f72588ddf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.057792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.058153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.058680 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.059689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:09.060319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.062320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.075282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 968/1931] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:09.076139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.087752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b89387-95a7-40ea-875e-164399c936ea tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.094812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.094812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.094812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.094812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.095205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.098652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.114082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 969/1932] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:09.114943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.125104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91de1c5-a857-4d98-87b1-dc17e175f70e admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.130094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.130981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.131268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.131577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] PATH_INFO: `/v3/projects/c65fc0532d7f4fd8985275ec02e74bf4/users/21cde9833b6e4a69a130d6c9b57ec712/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.148237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65fc0532d7f4fd8985275ec02e74bf4, user_id=21cde9833b6e4a69a130d6c9b57ec712, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.150942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.198264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 964/1933] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:09 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:09.208220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.232060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713ea44-65af-463e-a5d3-41be400951c8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.236538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ea07e332a4c3440ea7e4083465192a07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1d6b1f3afb864b79aea1b44f72588ddf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.249423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.249724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.250008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] PATH_INFO: `/v3/projects/afb0f29fa149468885e04fdfc2af2fb5` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.252380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorizing `identity:delete_project(project_id=afb0f29fa149468885e04fdfc2af2fb5)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.256082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.262873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 970/1934] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:09 2022] PUT /identity/v3/projects/c65fc0532d7f4fd8985275ec02e74bf4/users/21cde9833b6e4a69a130d6c9b57ec712/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:09.263835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.338247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327efeb-f3dd-4e86-8f88-db11fb31ced5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.344217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.345048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.345477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.345810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.346562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.348261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.371766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 971/1935] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:09.371766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.384642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c1126-3a3d-4208-8530-f14e707a3103 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.389607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.390309 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.390597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.390876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.391723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.393512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.405387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 972/1936] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:09.405557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.417041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44944dc1-c044-4261-8be5-3108c9ed0bb0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.423057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb0f29fa149468885e04fdfc2af2fb5', 'request_id': 'req-ac96c825-2392-49a9-94f8-1be47b014bb2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:09.475548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.476603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.476891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.477165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] PATH_INFO: `/v3/projects/08ebcb0566e84d409d849c7535b72797/users/b686b7d7240a484db08e2fa15380ed9c/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.493457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ebcb0566e84d409d849c7535b72797, user_id=b686b7d7240a484db08e2fa15380ed9c, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.496492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.535759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb0f29fa149468885e04fdfc2af2fb5', 'request_id': 'req-ac96c825-2392-49a9-94f8-1be47b014bb2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:09.544614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb0f29fa149468885e04fdfc2af2fb5', 'request_id': 'req-ac96c825-2392-49a9-94f8-1be47b014bb2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:09.558012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] The token cache is being invalidate because project afb0f29fa149468885e04fdfc2af2fb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:09.558294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:09.558980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:09.560747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 965/1937] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:09 2022] DELETE /identity/v3/projects/afb0f29fa149468885e04fdfc2af2fb5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:09.561743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.563689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:09.585641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96c825-2392-49a9-94f8-1be47b014bb2 tempest-VolumeQuotaClassesTest-1993648033 tempest-VolumeQuotaClassesTest-1993648033-project-admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.593707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.594935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.595286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.596304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] PATH_INFO: `/v3/projects/c65fc0532d7f4fd8985275ec02e74bf4/users/21cde9833b6e4a69a130d6c9b57ec712/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.598990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c65fc0532d7f4fd8985275ec02e74bf4, user_id=21cde9833b6e4a69a130d6c9b57ec712, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.602233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.604516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 973/1938] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:09 2022] PUT /identity/v3/projects/08ebcb0566e84d409d849c7535b72797/users/b686b7d7240a484db08e2fa15380ed9c/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:09.613463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.613463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:09.621345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 966/1939] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:09 2022] PUT /identity/v3/projects/c65fc0532d7f4fd8985275ec02e74bf4/users/21cde9833b6e4a69a130d6c9b57ec712/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:09.622557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.633388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:09.712209 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b66e5-c8b0-4dff-b362-9c4f4a1e556b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.715964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f114d2bf-3c20-465f-b754-8efa44760efc admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.717149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556332f-3553-4940-9e48-35b60edd0e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.717860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556332f-3553-4940-9e48-35b60edd0e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.718141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556332f-3553-4940-9e48-35b60edd0e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.718415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556332f-3553-4940-9e48-35b60edd0e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.719145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556332f-3553-4940-9e48-35b60edd0e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.720768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556332f-3553-4940-9e48-35b60edd0e3c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.722592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.723630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.724063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.724405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] PATH_INFO: `/v3/projects/a945d8008e5340889485c0a852485ba4` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.728313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a945d8008e5340889485c0a852485ba4)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.728762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.731355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 967/1940] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:09.732649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef2392-b711-4ac7-ad7e-c607d1a81e87 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.732944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef2392-b711-4ac7-ad7e-c607d1a81e87 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.733217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef2392-b711-4ac7-ad7e-c607d1a81e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.764099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-80ef2392-b711-4ac7-ad7e-c607d1a81e87 None None] MFA Rules not processed for user `21cde9833b6e4a69a130d6c9b57ec712`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:09.768438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-80ef2392-b711-4ac7-ad7e-c607d1a81e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:09.841031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 968/1941] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:09.842112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef2392-b711-4ac7-ad7e-c607d1a81e87 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.853914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef2392-b711-4ac7-ad7e-c607d1a81e87 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.858412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.861708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.861999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.862329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] PATH_INFO: `/v3/projects/08ebcb0566e84d409d849c7535b72797/users/b686b7d7240a484db08e2fa15380ed9c/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:09.865983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ebcb0566e84d409d849c7535b72797, user_id=b686b7d7240a484db08e2fa15380ed9c, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:09.867410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:09.878353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 969/1942] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:09 2022] PUT /identity/v3/projects/08ebcb0566e84d409d849c7535b72797/users/b686b7d7240a484db08e2fa15380ed9c/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:09.882703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:09.884436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:09.905016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a945d8008e5340889485c0a852485ba4', 'request_id': 'req-e6451808-35d0-41b6-aaaa-61a5c0c7f685'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:09.945435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078c7b1-e125-4a5e-aee0-247247ab350f admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:09.949348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:09.949993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:09.950296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:09.950570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:10.014448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:10.015860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:10.026821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 970/1943] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:10.037985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a945d8008e5340889485c0a852485ba4', 'request_id': 'req-e6451808-35d0-41b6-aaaa-61a5c0c7f685'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.046054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a945d8008e5340889485c0a852485ba4', 'request_id': 'req-e6451808-35d0-41b6-aaaa-61a5c0c7f685'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.058250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] The token cache is being invalidate because project a945d8008e5340889485c0a852485ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:10.058850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.059625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.061749 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 974/1944] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:09 2022] DELETE /identity/v3/projects/a945d8008e5340889485c0a852485ba4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:10.066750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:10.146220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3447ab-6ce0-4045-8187-672350ae616c service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:10.152592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:10.154010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:10.155488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:10.155488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] PATH_INFO: `/v3/users/7d0dc56d450c43aebd29afc11613331e` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:10.158492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d0dc56d450c43aebd29afc11613331e)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:10.161301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:10.426006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d0dc56d450c43aebd29afc11613331e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.426957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d0dc56d450c43aebd29afc11613331e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.436616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d0dc56d450c43aebd29afc11613331e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.437417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d0dc56d450c43aebd29afc11613331e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.450729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d0dc56d450c43aebd29afc11613331e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:10.654947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 971/1945] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:10 2022] DELETE /identity/v3/users/7d0dc56d450c43aebd29afc11613331e => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:10.985126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:10.987357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:11.075246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6451808-35d0-41b6-aaaa-61a5c0c7f685 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:11.116636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:11.118129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:11.166519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:11.167787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:11.167787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:11.167787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:11.174632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:11.213809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51608a08-cf95-4343-bf7b-4576d7f91f44 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:11.220029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:11.222243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:11.222243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:11.222243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] PATH_INFO: `/v3/users/cc60dc79dee140118712b9f7e4d58fba` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:11.223176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc60dc79dee140118712b9f7e4d58fba)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:11.225320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:11.290092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:11.291549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:11.302079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 975/1946] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:10 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:11.336510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:11.337973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:11.354164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb14a9-65cf-4f70-a351-cc360fed4777 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:11.361794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:11.362858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:11.363143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:11.363416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] PATH_INFO: `/v3/projects/d39b0ab1d56141f0ba2c54a3ae528373` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:11.364658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d39b0ab1d56141f0ba2c54a3ae528373)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:11.366648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:11.626658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd39b0ab1d56141f0ba2c54a3ae528373', 'request_id': 'req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.632186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc60dc79dee140118712b9f7e4d58fba'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.633006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc60dc79dee140118712b9f7e4d58fba'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.640139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc60dc79dee140118712b9f7e4d58fba'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.640826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc60dc79dee140118712b9f7e4d58fba'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.659480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-55af577c-daa4-46e9-9817-6d91c27d7384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc60dc79dee140118712b9f7e4d58fba'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.730089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd39b0ab1d56141f0ba2c54a3ae528373', 'request_id': 'req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.741337 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd39b0ab1d56141f0ba2c54a3ae528373', 'request_id': 'req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.756459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] The token cache is being invalidate because project d39b0ab1d56141f0ba2c54a3ae528373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:11.756965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.757816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-70b8fb52-b5a9-4a2f-ae8a-cba1f4e3ba29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:11.763765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 976/1947] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:11 2022] DELETE /identity/v3/projects/d39b0ab1d56141f0ba2c54a3ae528373 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:11.766390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d932e0-dae0-4509-a68d-d04f9ab54055 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:11.766832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d932e0-dae0-4509-a68d-d04f9ab54055 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:11.767772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d932e0-dae0-4509-a68d-d04f9ab54055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:11.790388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-77d932e0-dae0-4509-a68d-d04f9ab54055 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:11.801414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-77d932e0-dae0-4509-a68d-d04f9ab54055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:11.856080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 972/1948] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:11 2022] DELETE /identity/v3/users/cc60dc79dee140118712b9f7e4d58fba => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:11.861903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:11.864660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:11.865191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:11.869875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 977/1949] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:11.881588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15911aa-8eb7-418a-9101-3af93a8a4417 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:11.881879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15911aa-8eb7-418a-9101-3af93a8a4417 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:11.894725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15911aa-8eb7-418a-9101-3af93a8a4417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:11.894908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:11.894908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:11.902262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a15911aa-8eb7-418a-9101-3af93a8a4417 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:11.903509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a15911aa-8eb7-418a-9101-3af93a8a4417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:11.969140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 973/1950] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:11.975688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:11.979726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:11.982446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:12.026192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 978/1951] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:12.027537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.027840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.028099 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.075639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac817c59-9d2b-43eb-8f4a-41b1828a6430 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.076094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.076759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.077043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.077377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.091840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.094754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.096347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] MFA Rules not processed for user `b686b7d7240a484db08e2fa15380ed9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:12.110135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 974/1952] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:12.111694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.112336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:12.123699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d8ec5-3cba-4eb4-9027-375a6afc3af1 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.130293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.131041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.131327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.131680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.132420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:12.132773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.135760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.161694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 975/1953] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:12.162897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.181025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383b6b7-9e31-4b83-a3e6-019ed0bd7ab7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.187517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.188189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.188529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.188870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.189821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.192904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.231087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:12.272901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 979/1954] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:12.289554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.292224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 976/1955] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:12 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:12.293212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:12.298981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.368717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f56a-ad0b-4a3d-903e-8717efbd5500 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.412833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7029c3-ba9c-4e49-912b-e1662f49f51b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.419816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.420503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.420785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.421063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.422315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.424333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.450407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.451619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.458107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.458107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.467116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:12.525186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 977/1956] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:12 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:12.528885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.549656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3479d70-f078-41ab-85ca-da76077d4237 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.556234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.556387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.557239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.557913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.558241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.558346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.559093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.560907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.570298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 980/1957] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:12.574998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 978/1958] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:12.580345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.605792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcd9d3-9f45-4fc4-9073-e92473902d2e service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.611374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.612103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.612394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.612676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.614188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.616382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.631195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 981/1959] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:12.632831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.643823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1fb3d-2b35-4cc9-9884-e07cd1a5845e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.649096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abffbb5f-d4ab-45c4-93a1-ef0d93a8de7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.650195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffbb5f-d4ab-45c4-93a1-ef0d93a8de7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.650482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffbb5f-d4ab-45c4-93a1-ef0d93a8de7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.650761 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abffbb5f-d4ab-45c4-93a1-ef0d93a8de7f admin admin] PATH_INFO: `/v3/projects/6cc1545b83bd46a0a9393582f2f828bc/users/4f8ff90417374e99a78d49072f5895f1/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.673032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffbb5f-d4ab-45c4-93a1-ef0d93a8de7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc1545b83bd46a0a9393582f2f828bc, user_id=4f8ff90417374e99a78d49072f5895f1, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.676555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abffbb5f-d4ab-45c4-93a1-ef0d93a8de7f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.719561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:12.722032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:12.734805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064c4c5-d137-4ed0-9139-93ca52a74c34 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:12.740381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:12.741307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.741598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.741875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] PATH_INFO: `/v3/projects/01cf7d8d9d06491d8eda6230055967da` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.743082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01cf7d8d9d06491d8eda6230055967da)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:12.745005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:12.841035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 979/1960] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:12 2022] PUT /identity/v3/projects/6cc1545b83bd46a0a9393582f2f828bc/users/4f8ff90417374e99a78d49072f5895f1/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:12.848776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da34c-04ce-4115-8ccd-c56688ce229b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:12.849066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da34c-04ce-4115-8ccd-c56688ce229b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:12.849377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960da34c-04ce-4115-8ccd-c56688ce229b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:12.882051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-960da34c-04ce-4115-8ccd-c56688ce229b None None] MFA Rules not processed for user `4f8ff90417374e99a78d49072f5895f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:12.882051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-960da34c-04ce-4115-8ccd-c56688ce229b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:12.944963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01cf7d8d9d06491d8eda6230055967da', 'request_id': 'req-118f7ca2-193d-463e-837d-7ebcbba1cf20'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:12.969552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 980/1961] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:12.976266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960da34c-04ce-4115-8ccd-c56688ce229b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:13.052051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960da34c-04ce-4115-8ccd-c56688ce229b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:13.067877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01cf7d8d9d06491d8eda6230055967da', 'request_id': 'req-118f7ca2-193d-463e-837d-7ebcbba1cf20'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.083155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01cf7d8d9d06491d8eda6230055967da', 'request_id': 'req-118f7ca2-193d-463e-837d-7ebcbba1cf20'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.109724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] The token cache is being invalidate because project 01cf7d8d9d06491d8eda6230055967da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:13.110059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.115092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.117133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 982/1962] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:12 2022] DELETE /identity/v3/projects/01cf7d8d9d06491d8eda6230055967da => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:13.121545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:13.123032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:13.123473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:13.124140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.124974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.125532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:13.126428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:13.129058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:13.138165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f7ca2-193d-463e-837d-7ebcbba1cf20 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:13.168820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:13.169855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.170134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.170509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] PATH_INFO: `/v3/users/ea07e332a4c3440ea7e4083465192a07` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:13.172047 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea07e332a4c3440ea7e4083465192a07)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:13.176421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:13.256844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 981/1963] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:12 2022] POST /identity/v3/projects => generated 369 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:13.257982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:13.266066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:13.354716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab0ded-8f53-4531-9247-df6593d1ea99 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:13.362329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:13.363107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.363705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.364093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:13.364817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:13.366912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:13.429875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea07e332a4c3440ea7e4083465192a07'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.437222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea07e332a4c3440ea7e4083465192a07'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.437975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea07e332a4c3440ea7e4083465192a07'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.449972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea07e332a4c3440ea7e4083465192a07'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.472077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 982/1964] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:13 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:13.473447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:13.493095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba4d7c-cc3a-4841-958d-8d5a98209b31 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:13.510640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:13.511370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.512318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.512318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:13.512773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:13.515657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:13.526331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 983/1965] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:13.530394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:13.535176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea07e332a4c3440ea7e4083465192a07'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:13.546267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 983/1966] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:13 2022] DELETE /identity/v3/users/ea07e332a4c3440ea7e4083465192a07 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:13.546410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd445bf-2152-4715-aea8-796f66ffd3d3 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:13.635030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:13.635904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.636185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.636458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] PATH_INFO: `/v3/projects/ca721a16944944529d20d6deeaea49f4/users/7900ccd774024b1fa5d3d20bea271b2a/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:13.660736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca721a16944944529d20d6deeaea49f4, user_id=7900ccd774024b1fa5d3d20bea271b2a, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:13.663614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:13.793449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:13.793986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 984/1967] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:13 2022] PUT /identity/v3/projects/ca721a16944944529d20d6deeaea49f4/users/7900ccd774024b1fa5d3d20bea271b2a/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:13.794931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:13.809933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555352e-460e-4e86-873a-e046fe3940cd admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:13.886399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcb1d8-6a53-45bf-bb06-8a2635153e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:13.887093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb1d8-6a53-45bf-bb06-8a2635153e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.887376 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb1d8-6a53-45bf-bb06-8a2635153e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.887710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb1d8-6a53-45bf-bb06-8a2635153e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:13.888445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcb1d8-6a53-45bf-bb06-8a2635153e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:13.890135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcb1d8-6a53-45bf-bb06-8a2635153e23 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:13.900614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 984/1968] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:13.904875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:13.916515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b11036-c64f-4285-9eab-847e64b9feb2 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:13.921193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:13.922367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.922649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.923137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] PATH_INFO: `/v3/projects/ca721a16944944529d20d6deeaea49f4/users/7900ccd774024b1fa5d3d20bea271b2a/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:13.926952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca721a16944944529d20d6deeaea49f4, user_id=7900ccd774024b1fa5d3d20bea271b2a, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:13.929831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:13.949311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 985/1969] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:13 2022] PUT /identity/v3/projects/ca721a16944944529d20d6deeaea49f4/users/7900ccd774024b1fa5d3d20bea271b2a/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:13.955517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:13.955838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:13.956083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.000644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] MFA Rules not processed for user `7900ccd774024b1fa5d3d20bea271b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:14.002656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.085720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 985/1970] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:14.098358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:14.100153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.115104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:14.117145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.179584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7e620-9eb5-42b8-8d52-194f41bc90e9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:14.186398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:14.187107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.187391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.187723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.207255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:14.208798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:14.231610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 986/1971] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:14.260839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964c3cb-2edc-4833-ba1c-068d08808d22 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:14.268696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:14.269795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.270079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.270422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] PATH_INFO: `/v3/projects/1d6b1f3afb864b79aea1b44f72588ddf` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.274189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d6b1f3afb864b79aea1b44f72588ddf)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:14.276445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:14.374950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:14.376153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.389110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dec0ab1-1d68-492b-8dd7-2cee98b3cdba service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:14.392995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:14.393680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.393968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.394264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.396165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.461473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:14.462905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:14.473196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 987/1972] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:14.474167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:14.482529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb19427-ce23-4028-b805-3d3ef9f3882b service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:14.487540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f586df1-1a6c-4a4a-a968-a719c3cd654e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:14.488353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f586df1-1a6c-4a4a-a968-a719c3cd654e service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.488698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f586df1-1a6c-4a4a-a968-a719c3cd654e service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.489099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f586df1-1a6c-4a4a-a968-a719c3cd654e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.491032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4f586df1-1a6c-4a4a-a968-a719c3cd654e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.558771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d6b1f3afb864b79aea1b44f72588ddf', 'request_id': 'req-675325fe-da74-420f-9cc9-00b4d0ca1f69'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:14.567660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f586df1-1a6c-4a4a-a968-a719c3cd654e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:14.569093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f586df1-1a6c-4a4a-a968-a719c3cd654e service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:14.579125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 988/1973] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:14 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:14.609784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:14.639746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d6b1f3afb864b79aea1b44f72588ddf', 'request_id': 'req-675325fe-da74-420f-9cc9-00b4d0ca1f69'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:14.657519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d6b1f3afb864b79aea1b44f72588ddf', 'request_id': 'req-675325fe-da74-420f-9cc9-00b4d0ca1f69'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:14.674728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] The token cache is being invalidate because project 1d6b1f3afb864b79aea1b44f72588ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:14.675035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:14.676831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-675325fe-da74-420f-9cc9-00b4d0ca1f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:14.678755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 986/1974] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:14 2022] DELETE /identity/v3/projects/1d6b1f3afb864b79aea1b44f72588ddf => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:14.684111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3aac9-f75c-4206-894b-28e8cb65d3a3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.684781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3aac9-f75c-4206-894b-28e8cb65d3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.685336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3aac9-f75c-4206-894b-28e8cb65d3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.711315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e4d3aac9-f75c-4206-894b-28e8cb65d3a3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:14.713136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e4d3aac9-f75c-4206-894b-28e8cb65d3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.786740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 989/1975] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:14.794647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95016-175a-42e2-9173-323ffae52015 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.794935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95016-175a-42e2-9173-323ffae52015 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.795214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed95016-175a-42e2-9173-323ffae52015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.814925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-eed95016-175a-42e2-9173-323ffae52015 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:14.816075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-eed95016-175a-42e2-9173-323ffae52015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:14.825287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 987/1976] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:14.830471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3aac9-f75c-4206-894b-28e8cb65d3a3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:14.840185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3aac9-f75c-4206-894b-28e8cb65d3a3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:14.847700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:14.848368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.848647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.849118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.849893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:14.850353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:14.851954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:14.866095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 990/1977] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:14.870925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95016-175a-42e2-9173-323ffae52015 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:14.880339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed95016-175a-42e2-9173-323ffae52015 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:14.885091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:14.885785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:14.886067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:14.886342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:14.887060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:14.888997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:15.006298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 988/1978] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:14 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:15.010535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:15.027623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2d364-7d28-4bbe-9d30-b14901abc465 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:15.088722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:15.089540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:15.089865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:15.090330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:15.091112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:15.093127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:15.241511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 991/1979] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:15 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:15.246127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:15.262290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f0d4d-4ad9-48ce-a16d-d63922635f28 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:15.271341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:15.274713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:15.274713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:15.274713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:15.274713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:15.276547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:15.292717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 989/1980] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:15.294873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:15.308628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06080f-824c-406a-a6bf-9e9f1901ec8c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:15.318137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:15.318962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:15.319424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:15.319795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:15.320654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:15.322517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:15.346336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 992/1981] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:15.354180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:15.371326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16cbe4-aab2-4a0c-ba7e-b28352f693e5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:15.401131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c74a-7b34-45d7-92b5-ec8847b9ff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:15.401131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c74a-7b34-45d7-92b5-ec8847b9ff1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:15.401131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c74a-7b34-45d7-92b5-ec8847b9ff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:15.401131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c74a-7b34-45d7-92b5-ec8847b9ff1b admin admin] PATH_INFO: `/v3/projects/3efe80a1ba96448d9bd9fce671bd183a/users/10928ccb195146a48fd71a4b4dda74aa/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:15.409586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:15.411520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:15.427506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:15.446383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c74a-7b34-45d7-92b5-ec8847b9ff1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3efe80a1ba96448d9bd9fce671bd183a, user_id=10928ccb195146a48fd71a4b4dda74aa, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:15.455686 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c74a-7b34-45d7-92b5-ec8847b9ff1b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:15.527953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b809e2-4064-477c-b8ca-e6e67de0c8aa admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:15.532044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:15.532794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:15.533084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:15.533391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:15.535869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:15.591691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 990/1982] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:15 2022] PUT /identity/v3/projects/3efe80a1ba96448d9bd9fce671bd183a/users/10928ccb195146a48fd71a4b4dda74aa/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:15.603848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f01310-6f34-4266-9471-9924fe2ced16 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:15.604327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f01310-6f34-4266-9471-9924fe2ced16 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:15.604986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f01310-6f34-4266-9471-9924fe2ced16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:15.619617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:15.623039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:15.635554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 993/1983] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:15.644516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-68f01310-6f34-4266-9471-9924fe2ced16 None None] MFA Rules not processed for user `10928ccb195146a48fd71a4b4dda74aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:15.648370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-68f01310-6f34-4266-9471-9924fe2ced16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:15.714296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 991/1984] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:15.718904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:15.730463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef9291-6f3d-4cd2-a3bc-2c975fc5fb62 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:15.789884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:15.790460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:15.790803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:15.791090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:15.801859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:15.801859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:15.942749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 994/1985] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:15 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:15.947543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f01310-6f34-4266-9471-9924fe2ced16 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:15.960159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f01310-6f34-4266-9471-9924fe2ced16 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.018900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.019589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.019878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.020163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.020851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.022824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:16.136198 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 992/1986] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:15 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:16.145636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:16.156214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893fa3b-5b7e-4744-a408-78600b4dbcee admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.161542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.162341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.162624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.162967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.163739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.165655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:16.179845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 995/1987] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:16.183552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:16.194331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2fbd7-0fe3-4ec7-b645-9b976b64164a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.199086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f3a00-ad2c-42b2-b724-3ed7aedc95b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.199848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3a00-ad2c-42b2-b724-3ed7aedc95b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.200208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3a00-ad2c-42b2-b724-3ed7aedc95b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.200591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f3a00-ad2c-42b2-b724-3ed7aedc95b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.201361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3a00-ad2c-42b2-b724-3ed7aedc95b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.202962 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f3a00-ad2c-42b2-b724-3ed7aedc95b1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:16.213809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 993/1988] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:16.217916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:16.228943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8820bb22-bf1d-44dc-8e44-9eed5cd16003 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.233774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.234660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.234947 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.235268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] PATH_INFO: `/v3/projects/697e3e78c05749c2b1e92d6f0affac43/users/b3453ce50bcd4b969a740473aae89513/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.251165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=697e3e78c05749c2b1e92d6f0affac43, user_id=b3453ce50bcd4b969a740473aae89513, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.254161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:16.364223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 996/1989] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:16 2022] PUT /identity/v3/projects/697e3e78c05749c2b1e92d6f0affac43/users/b3453ce50bcd4b969a740473aae89513/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:16.377508 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946030b7-b2f3-4d56-b75a-27e3dd04a0eb None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.378007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946030b7-b2f3-4d56-b75a-27e3dd04a0eb None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.378100 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946030b7-b2f3-4d56-b75a-27e3dd04a0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.407552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-946030b7-b2f3-4d56-b75a-27e3dd04a0eb None None] MFA Rules not processed for user `b3453ce50bcd4b969a740473aae89513`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:16.407552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-946030b7-b2f3-4d56-b75a-27e3dd04a0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:16.471569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 994/1990] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:16.476318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:16.540812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844af077-e33c-43e9-a090-8b375ddf4b16 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.546385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.547555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.547838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.548112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.549161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.551408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:16.666010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 997/1991] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:16 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:16.669632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946030b7-b2f3-4d56-b75a-27e3dd04a0eb None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:16.683996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946030b7-b2f3-4d56-b75a-27e3dd04a0eb None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.750826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.752155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.752439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.752716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.753480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.755396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:16.882396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 995/1992] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:16 2022] POST /identity/v3/users => generated 299 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:16.886556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:16.897746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4560f7-1e5c-41f5-8d05-1eb2ae4a5a4b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.902179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.902853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.903132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.903417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.904110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.906015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:16.920061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 998/1993] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:16.924452 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:16.934529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104709ab-9b70-4f0e-a7dc-992fd88d6c79 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:16.939626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:16.940514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:16.940789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:16.941065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] PATH_INFO: `/v3/projects/0212bfb958b04ff4b9e424e22248fdbc/users/e489c7c99fd3443c93b48595ab35acbd/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:16.956544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0212bfb958b04ff4b9e424e22248fdbc, user_id=e489c7c99fd3443c93b48595ab35acbd, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:16.959202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:17.082230 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 996/1994] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:16 2022] PUT /identity/v3/projects/0212bfb958b04ff4b9e424e22248fdbc/users/e489c7c99fd3443c93b48595ab35acbd/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:17.086185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:17.096068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f811-359f-446e-847f-0a7695efb437 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:17.151699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25e299b-1651-425f-b3b6-f57192ae5aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:17.152369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e299b-1651-425f-b3b6-f57192ae5aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:17.152651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e299b-1651-425f-b3b6-f57192ae5aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:17.152944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25e299b-1651-425f-b3b6-f57192ae5aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:17.153695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e299b-1651-425f-b3b6-f57192ae5aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:17.155283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25e299b-1651-425f-b3b6-f57192ae5aa8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:17.167472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 999/1995] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:17.172929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:17.184688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb84fc-99ef-4135-a086-1a4480e37b4f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:17.190148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:17.191102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:17.191393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:17.191671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] PATH_INFO: `/v3/projects/0212bfb958b04ff4b9e424e22248fdbc/users/e489c7c99fd3443c93b48595ab35acbd/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:17.194661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0212bfb958b04ff4b9e424e22248fdbc, user_id=e489c7c99fd3443c93b48595ab35acbd, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:17.197418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:17.209121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 997/1996] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:17 2022] PUT /identity/v3/projects/0212bfb958b04ff4b9e424e22248fdbc/users/e489c7c99fd3443c93b48595ab35acbd/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:17.219164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f73779-659d-4fac-82cf-4749de2fe7a3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:17.219455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f73779-659d-4fac-82cf-4749de2fe7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:17.219738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f73779-659d-4fac-82cf-4749de2fe7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:17.249995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f3f73779-659d-4fac-82cf-4749de2fe7a3 None None] MFA Rules not processed for user `e489c7c99fd3443c93b48595ab35acbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:17.252008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f3f73779-659d-4fac-82cf-4749de2fe7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:17.324749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1000/1997] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:17.335867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:17.337758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:17.363925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:17.396409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8ff024-51b2-496b-9793-fd83f090f90d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:17.400097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:17.400775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:17.401056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:17.401399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:17.410494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:17.411918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:17.421466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 998/1998] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:17.540481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:19.043709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:19.319842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:19.344628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f73779-659d-4fac-82cf-4749de2fe7a3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:19.354530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f73779-659d-4fac-82cf-4749de2fe7a3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:19.357571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff1883-22c4-454f-bdb4-1c05037426fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:19.358218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff1883-22c4-454f-bdb4-1c05037426fb service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:19.358622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff1883-22c4-454f-bdb4-1c05037426fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:19.358902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff1883-22c4-454f-bdb4-1c05037426fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:19.418816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff1883-22c4-454f-bdb4-1c05037426fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:19.420301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff1883-22c4-454f-bdb4-1c05037426fb service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:19.431001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1001/1999] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:20.998002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:22.045101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:22.046598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:22.128638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87636c1-0c4e-45a9-9d17-9538b692ed08 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:22.133509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:22.134394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:22.134716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:22.134997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] PATH_INFO: `/v3/users/4f8ff90417374e99a78d49072f5895f1` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:22.136204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f8ff90417374e99a78d49072f5895f1)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:22.138215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:22.324207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:22.438801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f8ff90417374e99a78d49072f5895f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:22.445722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f8ff90417374e99a78d49072f5895f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:22.446408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f8ff90417374e99a78d49072f5895f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:22.459234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f8ff90417374e99a78d49072f5895f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:22.601129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f8ff90417374e99a78d49072f5895f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:22.607626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 999/2000] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:22 2022] DELETE /identity/v3/users/4f8ff90417374e99a78d49072f5895f1 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:22.612883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:22.613384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:22.613477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:22.631833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:22.633030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:22.690491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1002/2001] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:22.700027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:22.701453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:22.763234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43916b00-221d-4afa-b12f-8ec52088afe9 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:22.767810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:22.769017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:22.769017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:22.769017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:22.783676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:22.786097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:22.799084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1000/2002] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:22.809445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:23.296601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:23.298512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:23.310934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba00374-1282-488b-8af4-de7e93cb2c03 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:23.315848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:23.316725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:23.317020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:23.317328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] PATH_INFO: `/v3/projects/6cc1545b83bd46a0a9393582f2f828bc` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:23.318527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc1545b83bd46a0a9393582f2f828bc)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:23.320970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:23.541566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc1545b83bd46a0a9393582f2f828bc', 'request_id': 'req-7d16a94b-298a-4647-abc2-cf91429e31cc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:23.643293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc1545b83bd46a0a9393582f2f828bc', 'request_id': 'req-7d16a94b-298a-4647-abc2-cf91429e31cc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:23.649537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc1545b83bd46a0a9393582f2f828bc', 'request_id': 'req-7d16a94b-298a-4647-abc2-cf91429e31cc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:23.661970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] The token cache is being invalidate because project 6cc1545b83bd46a0a9393582f2f828bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:23.662270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:23.662958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:23.664748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1003/2003] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:23 2022] DELETE /identity/v3/projects/6cc1545b83bd46a0a9393582f2f828bc => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:23.668734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:23.678950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed4d94-6931-4918-bd48-e424255c6f99 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:23.742433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:23.743470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:23.743842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:23.744330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] PATH_INFO: `/v3/users/7900ccd774024b1fa5d3d20bea271b2a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:23.745664 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7900ccd774024b1fa5d3d20bea271b2a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:23.747588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:24.105170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7900ccd774024b1fa5d3d20bea271b2a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:24.113013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7900ccd774024b1fa5d3d20bea271b2a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:24.113755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7900ccd774024b1fa5d3d20bea271b2a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:24.127202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7900ccd774024b1fa5d3d20bea271b2a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:24.254960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7900ccd774024b1fa5d3d20bea271b2a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:24.262499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1001/2004] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:23 2022] DELETE /identity/v3/users/7900ccd774024b1fa5d3d20bea271b2a => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:24.696688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:24.698317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:24.719029 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16a94b-298a-4647-abc2-cf91429e31cc admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:24.799813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:24.801461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:24.801750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:24.802030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] PATH_INFO: `/v3/users/ecfc5d6661004641b1a8c689cd960946` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:24.803684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecfc5d6661004641b1a8c689cd960946)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:24.805666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:24.837159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:24.838903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:24.852453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6d6eb-2506-48e2-80d7-dff90a4e122d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:24.857229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:24.858243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:24.858783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:24.859086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] PATH_INFO: `/v3/users/d60f42ba5bbf4abcaf442cbc24b0a89e` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:24.860581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d60f42ba5bbf4abcaf442cbc24b0a89e)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:24.862835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:25.186063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecfc5d6661004641b1a8c689cd960946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.186970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecfc5d6661004641b1a8c689cd960946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.197478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecfc5d6661004641b1a8c689cd960946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.198873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecfc5d6661004641b1a8c689cd960946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.216622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecfc5d6661004641b1a8c689cd960946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.228215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd60f42ba5bbf4abcaf442cbc24b0a89e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.236316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd60f42ba5bbf4abcaf442cbc24b0a89e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.237169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd60f42ba5bbf4abcaf442cbc24b0a89e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.255700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd60f42ba5bbf4abcaf442cbc24b0a89e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.529394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1004/2005] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:24 2022] DELETE /identity/v3/users/ecfc5d6661004641b1a8c689cd960946 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:25.530281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:25.531634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:25.597473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad22c70-ccf0-4a20-a61c-316ac148dc9b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:25.603020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:25.611554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:25.611554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:25.611554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] PATH_INFO: `/v3/projects/ca721a16944944529d20d6deeaea49f4` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:25.611554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca721a16944944529d20d6deeaea49f4)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:25.611554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:25.682813 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-bd89f5dd-ca74-49da-82da-1d88f0f3edd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd60f42ba5bbf4abcaf442cbc24b0a89e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.695412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1002/2006] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:24 2022] DELETE /identity/v3/users/d60f42ba5bbf4abcaf442cbc24b0a89e => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:25.695412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1b1f0-42d1-4507-9938-2ad5f3e7e203 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:25.695412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1b1f0-42d1-4507-9938-2ad5f3e7e203 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:25.695412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1b1f0-42d1-4507-9938-2ad5f3e7e203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:25.711651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-10d1b1f0-42d1-4507-9938-2ad5f3e7e203 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:25.712965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-10d1b1f0-42d1-4507-9938-2ad5f3e7e203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:25.774535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1003/2007] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:25.775835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:25.776124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:25.776407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:25.794491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:25.795787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:25.810265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca721a16944944529d20d6deeaea49f4', 'request_id': 'req-6030e696-1320-4dcd-877d-5115f78c6259'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.855075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1004/2008] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:25.855930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:25.857244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:25.915423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca721a16944944529d20d6deeaea49f4', 'request_id': 'req-6030e696-1320-4dcd-877d-5115f78c6259'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.918633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0005ea-4151-4085-9efb-5b317d00224c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:25.924327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca721a16944944529d20d6deeaea49f4', 'request_id': 'req-6030e696-1320-4dcd-877d-5115f78c6259'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.924745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db1a553-89f0-4de2-acd1-fb8157a62f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:25.926528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1a553-89f0-4de2-acd1-fb8157a62f9f service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:25.927539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1a553-89f0-4de2-acd1-fb8157a62f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:25.927539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db1a553-89f0-4de2-acd1-fb8157a62f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:25.937823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] The token cache is being invalidate because project ca721a16944944529d20d6deeaea49f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:25.940086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1a553-89f0-4de2-acd1-fb8157a62f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:25.940718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.941683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:25.945683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db1a553-89f0-4de2-acd1-fb8157a62f9f service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:25.945841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1005/2009] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:25 2022] DELETE /identity/v3/projects/ca721a16944944529d20d6deeaea49f4 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:25.945841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:25.946310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:25.958971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1005/2010] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:25.959799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6030e696-1320-4dcd-877d-5115f78c6259 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:25.960463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f665f09-969e-426e-a45f-2e624ee36467 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:25.960748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f665f09-969e-426e-a45f-2e624ee36467 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:25.961031 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f665f09-969e-426e-a45f-2e624ee36467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:25.964585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:25.965245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:25.965617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:25.965897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:25.967704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:25.986311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:25.986311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:25.986682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8f665f09-969e-426e-a45f-2e624ee36467 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:25.986682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8f665f09-969e-426e-a45f-2e624ee36467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:25.997036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1006/2012] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:25.997890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1006/2011] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:26.004925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf00a-0d7c-4a8a-bd95-5cc8d2ae90bd None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.005220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf00a-0d7c-4a8a-bd95-5cc8d2ae90bd None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.005567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdf00a-0d7c-4a8a-bd95-5cc8d2ae90bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.026735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1cbdf00a-0d7c-4a8a-bd95-5cc8d2ae90bd None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:26.028254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1cbdf00a-0d7c-4a8a-bd95-5cc8d2ae90bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:26.039228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:26.040838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1007/2013] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:26.041943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.052883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb462d3-f52d-419d-9542-b3dce23754f1 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.058682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.059356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.059815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.060257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.060985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:26.061824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.064215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.077564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1007/2014] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:14:26.082593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdf00a-0d7c-4a8a-bd95-5cc8d2ae90bd None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.093453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdf00a-0d7c-4a8a-bd95-5cc8d2ae90bd None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.105390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.105390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.105390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.105390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.105781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.112168 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.114512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.117320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:26.182267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678e5d4-e457-4c64-9a25-f8b340c5d4a5 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.186987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.186987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.187219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.187439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.189404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:26.253024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.255335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.267710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1008/2015] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:26.299397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1008/2016] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:26 2022] POST /identity/v3/projects => generated 387 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:26.303427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.328092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832bd7ba-32b6-4106-8217-c629e870b6be service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.405227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.406067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.406563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.406854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.409742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.410287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.544661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1009/2017] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:26 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:26.548934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.568202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ea53e-6abc-4a67-b7f0-63c707d76a9b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.575839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.576545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.576901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.577174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.577946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.581354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.590404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1009/2018] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:26.594411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.605334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708a98d-9106-4b60-8097-9b59652d355e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.610223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.611577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.611899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.612184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.612927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.614578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.624513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1010/2019] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:26.628910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.639036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef63a55-a741-4fc4-b71c-ad1d094f8c1b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.644622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.645111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.645431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.645714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] PATH_INFO: `/v3/projects/def22420c668481d8bd204e1a6056ceb/users/92797baeea7c4a8ea56d72c030b48406/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.650521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.652468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:26.666758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072896d-778b-4aa0-b41b-28ec88633d57 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.668710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] RBAC: Authorizing `identity:create_grant(project_id=def22420c668481d8bd204e1a6056ceb, user_id=92797baeea7c4a8ea56d72c030b48406, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.671579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.673853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.674505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.675046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.675412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] PATH_INFO: `/v3/projects/b821b9f24df0462ca04377c94d19a170` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.676870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b821b9f24df0462ca04377c94d19a170)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.678826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.693535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:26.761320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1010/2020] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:26 2022] PUT /identity/v3/projects/def22420c668481d8bd204e1a6056ceb/users/92797baeea7c4a8ea56d72c030b48406/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:26.765072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:26.766064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:26.811877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b821b9f24df0462ca04377c94d19a170', 'request_id': 'req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:26.834070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed457-ce41-4ebd-8b1b-675fd003bdff admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:26.877723 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b821b9f24df0462ca04377c94d19a170', 'request_id': 'req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:26.885368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b821b9f24df0462ca04377c94d19a170', 'request_id': 'req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:26.896424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:26.897491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.897846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.898235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] PATH_INFO: `/v3/projects/171b48aa7e3341bf8a9ea1b4e256ea4e` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.899567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171b48aa7e3341bf8a9ea1b4e256ea4e)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:26.900879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] The token cache is being invalidate because project b821b9f24df0462ca04377c94d19a170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:26.901390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:26.901718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:26.902182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ea6236aa-5cb1-4721-9c5b-2cd7d43e003e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:26.903976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1011/2021] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:26 2022] DELETE /identity/v3/projects/b821b9f24df0462ca04377c94d19a170 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:26.905310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:26.905660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:26.905949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:26.935892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] MFA Rules not processed for user `92797baeea7c4a8ea56d72c030b48406`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:26.937469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:27.010701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1012/2022] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:27.010701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.010701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:27.025553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ae2c6-335e-4132-9d73-cc215c83c039 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.030135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.031002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.031289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.031588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] PATH_INFO: `/v3/users/365f283179494b9d822e7befd4a677c9` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.033497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365f283179494b9d822e7befd4a677c9)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.035847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.060639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171b48aa7e3341bf8a9ea1b4e256ea4e', 'request_id': 'req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.197614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171b48aa7e3341bf8a9ea1b4e256ea4e', 'request_id': 'req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.214085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171b48aa7e3341bf8a9ea1b4e256ea4e', 'request_id': 'req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.226580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] The token cache is being invalidate because project 171b48aa7e3341bf8a9ea1b4e256ea4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:27.226916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.227595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.229475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1011/2023] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:26 2022] DELETE /identity/v3/projects/171b48aa7e3341bf8a9ea1b4e256ea4e => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:27.230303 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.231620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:27.291216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365f283179494b9d822e7befd4a677c9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.291894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365f283179494b9d822e7befd4a677c9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.295547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7b736-0cd5-4bc9-b8f3-914b9c1e6879 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.299713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365f283179494b9d822e7befd4a677c9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.300495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365f283179494b9d822e7befd4a677c9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.302302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.302980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.303349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.303631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.304335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.306285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.315356 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365f283179494b9d822e7befd4a677c9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.432244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1013/2024] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:27 2022] DELETE /identity/v3/users/365f283179494b9d822e7befd4a677c9 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:27.435245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.435245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:27.475433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1012/2025] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:27 2022] POST /identity/v3/projects => generated 391 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:27.476476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.478521 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:27.537005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e469a-3ee9-4ce5-9f75-7874c0420060 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.560444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898cba6-361d-4f87-8ba2-251963e84911 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.566687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.567592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.567877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.568153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.568857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.570849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.610591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.611311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.611849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.612137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] PATH_INFO: `/v3/users/21cde9833b6e4a69a130d6c9b57ec712` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.613440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21cde9833b6e4a69a130d6c9b57ec712)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.615405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.692276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1013/2026] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:27 2022] POST /identity/v3/users => generated 302 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:27.693395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.703039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79906446-ce93-4a13-bbb9-3d64a01693d2 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.707925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.708591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.708889 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.709150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.709908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.711514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.722088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1014/2027] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:27.726018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.736494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708d3ee-22ef-41ef-9c18-2afd2c6e5dde admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.741725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.742636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.742913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.743190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] PATH_INFO: `/v3/projects/1c35f8a71d2f4820b25a3ad321d5f71c/users/982dadd83d294b75a48f9f06172e8887/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.767671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c35f8a71d2f4820b25a3ad321d5f71c, user_id=982dadd83d294b75a48f9f06172e8887, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.767671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.851232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1015/2028] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:27 2022] PUT /identity/v3/projects/1c35f8a71d2f4820b25a3ad321d5f71c/users/982dadd83d294b75a48f9f06172e8887/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:27.855390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.861288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21cde9833b6e4a69a130d6c9b57ec712'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.862537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21cde9833b6e4a69a130d6c9b57ec712'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.869935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21cde9833b6e4a69a130d6c9b57ec712'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.870760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21cde9833b6e4a69a130d6c9b57ec712'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.883385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-75c0b861-cee8-41ee-ad38-64ff4c1e7c29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21cde9833b6e4a69a130d6c9b57ec712'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:27.920399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444e68d-e69d-4f27-a830-69129bb29bca admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.926363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.927080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.927368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.927649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.928437 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.930088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.941093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1016/2029] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:27.945049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.946777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:27.964528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d3543-d307-4ce1-b357-cfebfccadd72 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:27.970147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:27.971360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:27.971549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:27.971830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] PATH_INFO: `/v3/projects/1c35f8a71d2f4820b25a3ad321d5f71c/users/982dadd83d294b75a48f9f06172e8887/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:27.974097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c35f8a71d2f4820b25a3ad321d5f71c, user_id=982dadd83d294b75a48f9f06172e8887, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:27.976718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:27.992569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1017/2030] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:27 2022] PUT /identity/v3/projects/1c35f8a71d2f4820b25a3ad321d5f71c/users/982dadd83d294b75a48f9f06172e8887/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:27.994557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:27.999553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.029920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1014/2031] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:27 2022] DELETE /identity/v3/users/21cde9833b6e4a69a130d6c9b57ec712 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:28.033879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.033879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.033879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.072613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] MFA Rules not processed for user `982dadd83d294b75a48f9f06172e8887`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:28.077480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.078525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf269b-23d0-4f18-baa8-14229b2632d6 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:28.132997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:28.183080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1015/2032] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:28.191235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:28.195346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.195346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.195346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] PATH_INFO: `/v3/projects/d060b3c8349c4ef184bf8dad91bbc8a6` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.195346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d060b3c8349c4ef184bf8dad91bbc8a6)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:28.196401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:28.231706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:28.237991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.309799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc2c6c-aa0d-4023-a017-76793ebf12c2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:28.325732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:28.326406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.326700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.326985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.340788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd060b3c8349c4ef184bf8dad91bbc8a6', 'request_id': 'req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.348904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:28.350429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:28.361986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1016/2033] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:28.451716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:28.461190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb18bab5-10c6-496e-ad0d-63f1f975f4b8 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:28.489621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd060b3c8349c4ef184bf8dad91bbc8a6', 'request_id': 'req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.497189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd060b3c8349c4ef184bf8dad91bbc8a6', 'request_id': 'req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.510738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] The token cache is being invalidate because project d060b3c8349c4ef184bf8dad91bbc8a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:28.511045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.511650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-2d0071ca-d802-4fd1-a5d5-8d3f197c6d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.513511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1018/2034] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:27 2022] DELETE /identity/v3/projects/d060b3c8349c4ef184bf8dad91bbc8a6 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:28.514830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c60c72-2e38-4a9f-bcac-4ef8410b2bab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:28.515736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60c72-2e38-4a9f-bcac-4ef8410b2bab service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.516201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60c72-2e38-4a9f-bcac-4ef8410b2bab service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.517238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c60c72-2e38-4a9f-bcac-4ef8410b2bab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.519884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.520419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.520540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f3c60c72-2e38-4a9f-bcac-4ef8410b2bab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.520712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.544124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:28.545547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.628684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c60c72-2e38-4a9f-bcac-4ef8410b2bab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:28.636478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c60c72-2e38-4a9f-bcac-4ef8410b2bab service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:28.642636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1019/2035] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:28.642636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:28.646088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.654174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1017/2036] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:28.655661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651a84-0121-4161-bad7-d2d0d1b47117 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.655940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651a84-0121-4161-bad7-d2d0d1b47117 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.656222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9651a84-0121-4161-bad7-d2d0d1b47117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.663562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740b37de-be57-4ca9-853d-d7813b860e63 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:28.672216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:28.674335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.674617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.675088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] PATH_INFO: `/v3/projects/c65fc0532d7f4fd8985275ec02e74bf4` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.676675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c65fc0532d7f4fd8985275ec02e74bf4)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:28.678577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e9651a84-0121-4161-bad7-d2d0d1b47117 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:28.679958 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e9651a84-0121-4161-bad7-d2d0d1b47117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.680383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:28.692408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1018/2037] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:28.693862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9651a84-0121-4161-bad7-d2d0d1b47117 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:28.707001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9651a84-0121-4161-bad7-d2d0d1b47117 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:28.712987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:28.714595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.715234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.715614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.716463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:28.716872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:28.718625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:28.737312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1019/2038] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:28.739932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:28.752235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b1f17-912a-419e-a045-8cddb7514f8b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:28.757935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:28.758766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.759284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.759662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.760445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:28.762920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:28.836516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c65fc0532d7f4fd8985275ec02e74bf4', 'request_id': 'req-9ac3584a-312b-4159-b71e-ae77d14641b2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.894233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1020/2039] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:28 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:28.896296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:28.917558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c65fc0532d7f4fd8985275ec02e74bf4', 'request_id': 'req-9ac3584a-312b-4159-b71e-ae77d14641b2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.925779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c65fc0532d7f4fd8985275ec02e74bf4', 'request_id': 'req-9ac3584a-312b-4159-b71e-ae77d14641b2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.939831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] The token cache is being invalidate because project c65fc0532d7f4fd8985275ec02e74bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:28.940183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.940875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-9ac3584a-312b-4159-b71e-ae77d14641b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:28.943165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1020/2040] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:28 2022] DELETE /identity/v3/projects/c65fc0532d7f4fd8985275ec02e74bf4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:28.950067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0fb63-e5ff-4a95-b65f-94ec6865cc83 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.950353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0fb63-e5ff-4a95-b65f-94ec6865cc83 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.950853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0fb63-e5ff-4a95-b65f-94ec6865cc83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.971754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1dc0fb63-e5ff-4a95-b65f-94ec6865cc83 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:28.973293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1dc0fb63-e5ff-4a95-b65f-94ec6865cc83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:28.981568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1021/2041] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:28.981779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609343b-9406-4218-b76b-34226dae1ff4 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:28.989825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66030518-ce28-4f2a-9998-ca8565841b78 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.990178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66030518-ce28-4f2a-9998-ca8565841b78 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.990397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66030518-ce28-4f2a-9998-ca8565841b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.993123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:28.994207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:28.999388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:28.999388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:28.999388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:28.999388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.009375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-66030518-ce28-4f2a-9998-ca8565841b78 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:29.010587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-66030518-ce28-4f2a-9998-ca8565841b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:29.019683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1022/2042] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:29.023908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66030518-ce28-4f2a-9998-ca8565841b78 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.033114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66030518-ce28-4f2a-9998-ca8565841b78 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.037858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.038523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.038804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.039082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.039873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:29.040177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.041828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.054262 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1023/2043] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:14:29.058870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.069506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d34b71-c3cb-4420-9d53-171a9147d7a2 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.074573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.075703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.076545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.076812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.077794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.081302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.089212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1021/2044] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:28 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:29.093049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.096289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:29.110269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219647e-325b-463a-9974-4e51632732ab admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.118585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.119277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.120665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.120978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.122695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.124314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.141077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1022/2045] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:29.149148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.162730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde2f05-9b0c-4cd5-a19f-71b9419b0bb9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.172113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.172113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.172113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.172113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.172449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.174168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.186297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1023/2046] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:29.193850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.208636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfbcef-d12e-492e-8e63-44df0c95f54a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.212657 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043914d-1a53-4661-b85c-f571862df528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.215259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043914d-1a53-4661-b85c-f571862df528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.215580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043914d-1a53-4661-b85c-f571862df528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.215872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043914d-1a53-4661-b85c-f571862df528 admin admin] PATH_INFO: `/v3/projects/27756352b7514ef0b8622ab202a663a1/users/d52c4cbba49d459bb0fbed25f7cfda46/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.238655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1024/2047] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:29 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:29.241182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043914d-1a53-4661-b85c-f571862df528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27756352b7514ef0b8622ab202a663a1, user_id=d52c4cbba49d459bb0fbed25f7cfda46, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.242723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.244075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043914d-1a53-4661-b85c-f571862df528 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.306507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b163b-9a95-4747-b6fc-ea118a5a2fbf admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.311339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.312001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.312282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.312557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.313240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.315743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.335058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1024/2048] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:29 2022] PUT /identity/v3/projects/27756352b7514ef0b8622ab202a663a1/users/d52c4cbba49d459bb0fbed25f7cfda46/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:29.346959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ee74-877e-43f5-829f-20d683bc55d4 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.347247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ee74-877e-43f5-829f-20d683bc55d4 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.347548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77ee74-877e-43f5-829f-20d683bc55d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.375605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1d77ee74-877e-43f5-829f-20d683bc55d4 None None] MFA Rules not processed for user `d52c4cbba49d459bb0fbed25f7cfda46`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:29.377478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1d77ee74-877e-43f5-829f-20d683bc55d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:29.446468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1025/2049] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:29.494791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1025/2050] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:29 2022] POST /identity/v3/users => generated 292 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:29.496424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ee74-877e-43f5-829f-20d683bc55d4 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.503329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.505316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:29.568706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add60604-7927-4241-b2bd-6dd0ea79116d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.571514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77ee74-877e-43f5-829f-20d683bc55d4 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.577064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.577898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.578311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.578736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.579390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.581171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.593454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1026/2051] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:29.597395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.607788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3ee1c-b7fa-4bc4-87c1-62539239f4d1 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.612926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.613741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.614046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.614436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.615175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.617241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.621576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.622238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.622588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.629589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.632537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.634091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.636407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1027/2052] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:29.640481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.645188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1026/2053] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:29.652778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908927fa-9af5-4856-a3cb-4c03fd85c15b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.659407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8aaebc-1bea-4410-8d81-e69fb0f762db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.660269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aaebc-1bea-4410-8d81-e69fb0f762db admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.660552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aaebc-1bea-4410-8d81-e69fb0f762db admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.660828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8aaebc-1bea-4410-8d81-e69fb0f762db admin admin] PATH_INFO: `/v3/projects/7df9f3e832c0492682716b721e7dfde3/users/ccc2ed98e4df4615a380382d9b340bda/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.673549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.679178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8aaebc-1bea-4410-8d81-e69fb0f762db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df9f3e832c0492682716b721e7dfde3, user_id=ccc2ed98e4df4615a380382d9b340bda, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.681995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8aaebc-1bea-4410-8d81-e69fb0f762db admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.684370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabc0a4-d6b8-4aa9-bf26-4e0c1631046a service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.690632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.691513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.691791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.692062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] PATH_INFO: `/v3/users/d52c4cbba49d459bb0fbed25f7cfda46` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.693283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52c4cbba49d459bb0fbed25f7cfda46)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.695228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:29.803918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1028/2054] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:29 2022] PUT /identity/v3/projects/7df9f3e832c0492682716b721e7dfde3/users/ccc2ed98e4df4615a380382d9b340bda/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:29.827920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6bb9e-d447-4d6d-b7dc-ce1e332745a8 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.827920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6bb9e-d447-4d6d-b7dc-ce1e332745a8 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.827920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6bb9e-d447-4d6d-b7dc-ce1e332745a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.853355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1aa6bb9e-d447-4d6d-b7dc-ce1e332745a8 None None] MFA Rules not processed for user `ccc2ed98e4df4615a380382d9b340bda`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:29.853355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1aa6bb9e-d447-4d6d-b7dc-ce1e332745a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:29.917617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1029/2055] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:29.922584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6bb9e-d447-4d6d-b7dc-ce1e332745a8 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:29.986597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6bb9e-d447-4d6d-b7dc-ce1e332745a8 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:29.991164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:29.991842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:29.992105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:29.992379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:29.993067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:29.995338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:30.006787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52c4cbba49d459bb0fbed25f7cfda46'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:30.014886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52c4cbba49d459bb0fbed25f7cfda46'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:30.015597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52c4cbba49d459bb0fbed25f7cfda46'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:30.020030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:30.031969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52c4cbba49d459bb0fbed25f7cfda46'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:30.122197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8fb412fc-190b-426f-8e7a-e5107de8eea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52c4cbba49d459bb0fbed25f7cfda46'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:30.130768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1030/2056] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:29 2022] POST /identity/v3/projects => generated 379 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:30.131537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1027/2057] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:29 2022] DELETE /identity/v3/users/d52c4cbba49d459bb0fbed25f7cfda46 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:30.135347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:30.135842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.136271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.136645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.137604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:30.171315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:30.172144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:30.224542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c0719-2b63-46c2-9ee3-335315f80797 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:30.248315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1028/2058] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:30.262509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:30.264332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:30.299347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:30.302421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.302421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.302421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.302421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:30.306896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:30.342556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f77fd4d-c22e-499c-a4a6-a8f1e523876b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:30.346772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:30.348568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:30.349226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.349540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.349821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.359077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:30.361435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:30.371012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1029/2059] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:30.440591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1031/2060] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:30 2022] POST /identity/v3/users => generated 296 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:30.442182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:30.451961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f86932-82a0-49fd-9eea-9a6ae717b50a service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:30.458043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:30.458767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.459049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.459731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.460032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:30.461746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:30.473297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1030/2061] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:30.477986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:30.488986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d193e-8df8-451f-beb8-4ed8991b8291 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:30.493685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:30.494566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.494847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.495146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] PATH_INFO: `/v3/projects/53ad9d224c3941079ad98dad3a7cb531/users/88400114bf0642f48e7bdd8ecbd83761/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.513556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ad9d224c3941079ad98dad3a7cb531, user_id=88400114bf0642f48e7bdd8ecbd83761, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:30.513556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:30.647989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1032/2062] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:30 2022] PUT /identity/v3/projects/53ad9d224c3941079ad98dad3a7cb531/users/88400114bf0642f48e7bdd8ecbd83761/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:30.651850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:30.662601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50150f-ca1c-457d-a1eb-dd88d4f450af admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:30.723074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d18e5a4-2123-4eb7-968e-149d1507a227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:30.723830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18e5a4-2123-4eb7-968e-149d1507a227 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.725925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18e5a4-2123-4eb7-968e-149d1507a227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.725925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d18e5a4-2123-4eb7-968e-149d1507a227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.725925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18e5a4-2123-4eb7-968e-149d1507a227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:30.727548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d18e5a4-2123-4eb7-968e-149d1507a227 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:30.740449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1031/2063] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:30.744326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:30.755287 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff818-3480-43d5-b668-93ce85b6c529 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:30.760706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:30.761641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.767099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.767099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] PATH_INFO: `/v3/projects/53ad9d224c3941079ad98dad3a7cb531/users/88400114bf0642f48e7bdd8ecbd83761/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.767099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ad9d224c3941079ad98dad3a7cb531, user_id=88400114bf0642f48e7bdd8ecbd83761, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:30.767539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:30.787659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1033/2064] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:30 2022] PUT /identity/v3/projects/53ad9d224c3941079ad98dad3a7cb531/users/88400114bf0642f48e7bdd8ecbd83761/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:30.791433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:30.791809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:30.792148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:30.821576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] MFA Rules not processed for user `88400114bf0642f48e7bdd8ecbd83761`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:30.822878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:30.909215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1032/2065] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:31.034786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:31.036661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:31.068063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:31.069606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:31.107352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2f8dc-9e2a-4d22-8664-02b1b462c441 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:31.110044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:31.110771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:31.111048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:31.111600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:31.125614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:31.128783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:31.141331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1034/2066] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:31.141331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:31.146762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:31.165382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95543b-d7f6-41a6-b0f2-0e68ba69d337 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:31.167275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de93ecd-b62b-4773-ab37-5adf24cb1c97 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:31.170858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:31.172004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:31.172290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:31.172592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] PATH_INFO: `/v3/projects/27756352b7514ef0b8622ab202a663a1` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:31.172923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:31.174159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27756352b7514ef0b8622ab202a663a1)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:31.174286 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:31.174596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:31.174945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] PATH_INFO: `/v3/users/13f9b782a79f48678f297d1ef388da75` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:31.176091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:31.176274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13f9b782a79f48678f297d1ef388da75)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:31.178414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:31.424536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27756352b7514ef0b8622ab202a663a1', 'request_id': 'req-a7fc697d-b1e8-4532-8b6d-ee741b53c866'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.424746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13f9b782a79f48678f297d1ef388da75'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.425480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13f9b782a79f48678f297d1ef388da75'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.439196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13f9b782a79f48678f297d1ef388da75'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.439941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13f9b782a79f48678f297d1ef388da75'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.452603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-cbd07f76-331c-40d4-b31e-ff3e7d9f0461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13f9b782a79f48678f297d1ef388da75'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.562470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27756352b7514ef0b8622ab202a663a1', 'request_id': 'req-a7fc697d-b1e8-4532-8b6d-ee741b53c866'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.569329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27756352b7514ef0b8622ab202a663a1', 'request_id': 'req-a7fc697d-b1e8-4532-8b6d-ee741b53c866'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.580546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] The token cache is being invalidate because project 27756352b7514ef0b8622ab202a663a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:31.580947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.581752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7fc697d-b1e8-4532-8b6d-ee741b53c866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:31.583365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1033/2067] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:31 2022] DELETE /identity/v3/projects/27756352b7514ef0b8622ab202a663a1 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:31.588215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ff970-d578-4f2c-89b1-0d571a06545e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:31.588503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ff970-d578-4f2c-89b1-0d571a06545e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:31.588784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ff970-d578-4f2c-89b1-0d571a06545e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:31.615651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a49ff970-d578-4f2c-89b1-0d571a06545e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:31.616863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a49ff970-d578-4f2c-89b1-0d571a06545e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:31.659092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1035/2068] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:31 2022] DELETE /identity/v3/users/13f9b782a79f48678f297d1ef388da75 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:31.687019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1034/2069] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:31.694900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e1a16-e398-4bf1-b99a-8010b75b9a21 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:31.695660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e1a16-e398-4bf1-b99a-8010b75b9a21 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:31.696115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e1a16-e398-4bf1-b99a-8010b75b9a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:31.718249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8a3e1a16-e398-4bf1-b99a-8010b75b9a21 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:31.722058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8a3e1a16-e398-4bf1-b99a-8010b75b9a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:31.794829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1036/2070] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:31.799138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ff970-d578-4f2c-89b1-0d571a06545e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:31.809408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ff970-d578-4f2c-89b1-0d571a06545e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:31.815551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:31.816143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:31.816430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:31.816708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:31.817482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:31.817795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:31.819566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:31.833308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1035/2071] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:14:31.846955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e1a16-e398-4bf1-b99a-8010b75b9a21 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:31.858826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e1a16-e398-4bf1-b99a-8010b75b9a21 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:31.864271 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:31.864934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:31.865242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:31.865550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:31.866249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:31.868243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:31.990369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:31.990369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:31.991418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1037/2072] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:31 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:31.994012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.006854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2cf3f-3157-46f8-a3bf-eafa063bd3db admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.065324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c06560-9119-4043-af55-4692aa667598 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.070130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.070582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.070885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.070986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.071179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.071273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.071460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.071551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] PATH_INFO: `/v3/projects/4784aebbfd1d43f896df6ca9057b3251` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.072156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.072727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4784aebbfd1d43f896df6ca9057b3251)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.074163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.074656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.197299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1038/2073] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:31 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:32.198363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.205207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4784aebbfd1d43f896df6ca9057b3251', 'request_id': 'req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.211763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fad00b-2bd6-4ca4-8c25-7925b8aa0a61 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.266348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4784aebbfd1d43f896df6ca9057b3251', 'request_id': 'req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.269550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.270237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.274679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.274679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.274679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.275056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4784aebbfd1d43f896df6ca9057b3251', 'request_id': 'req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.275148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.286413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1039/2074] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:32.286413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.286727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] The token cache is being invalidate because project 4784aebbfd1d43f896df6ca9057b3251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:32.286727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.286727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.286971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.287830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1036/2075] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:31 2022] DELETE /identity/v3/projects/4784aebbfd1d43f896df6ca9057b3251 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:32.288902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.290263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.304204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c0ccb-32c8-4ffe-8eef-8f04bf580de3 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.308766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a67d2-2e33-4c36-bd2f-644e08705eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.309469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a67d2-2e33-4c36-bd2f-644e08705eba admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.309750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a67d2-2e33-4c36-bd2f-644e08705eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.310073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a67d2-2e33-4c36-bd2f-644e08705eba admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.310796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a67d2-2e33-4c36-bd2f-644e08705eba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.312405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a67d2-2e33-4c36-bd2f-644e08705eba admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.323350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1037/2076] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:32.324663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f72144-4dfb-4ef8-a43c-d8704fd43fe6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.324952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f72144-4dfb-4ef8-a43c-d8704fd43fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.325226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f72144-4dfb-4ef8-a43c-d8704fd43fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.346613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-10f72144-4dfb-4ef8-a43c-d8704fd43fe6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:32.347744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-10f72144-4dfb-4ef8-a43c-d8704fd43fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.349448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d20e8-c0ac-4f7c-a026-8e327b69d7dc admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.353392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e9e54-cf5d-41f3-bed8-6985d7495b7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.354107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e9e54-cf5d-41f3-bed8-6985d7495b7f service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.354484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e9e54-cf5d-41f3-bed8-6985d7495b7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.354974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e9e54-cf5d-41f3-bed8-6985d7495b7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.357078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c69e9e54-cf5d-41f3-bed8-6985d7495b7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.359436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1038/2077] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:32.372533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f72144-4dfb-4ef8-a43c-d8704fd43fe6 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.372533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f72144-4dfb-4ef8-a43c-d8704fd43fe6 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.381950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16112dd7-0846-412d-88ba-ae9b0bc2377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.382866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112dd7-0846-412d-88ba-ae9b0bc2377f admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.383158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112dd7-0846-412d-88ba-ae9b0bc2377f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.383439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112dd7-0846-412d-88ba-ae9b0bc2377f admin admin] PATH_INFO: `/v3/projects/3db6db3913f64c9f8a1b0aba7bd9524f/users/f16805a7a6994701aaac66eacb94b6c6/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.400851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16112dd7-0846-412d-88ba-ae9b0bc2377f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db6db3913f64c9f8a1b0aba7bd9524f, user_id=f16805a7a6994701aaac66eacb94b6c6, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.403621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16112dd7-0846-412d-88ba-ae9b0bc2377f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.426285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e9e54-cf5d-41f3-bed8-6985d7495b7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.427754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e9e54-cf5d-41f3-bed8-6985d7495b7f service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.437701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1040/2078] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:32.439240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec554a7-e5fc-44b3-bedf-46c64ad8be1f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.439561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec554a7-e5fc-44b3-bedf-46c64ad8be1f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.439839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec554a7-e5fc-44b3-bedf-46c64ad8be1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.460183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8ec554a7-e5fc-44b3-bedf-46c64ad8be1f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:32.460511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8ec554a7-e5fc-44b3-bedf-46c64ad8be1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.468433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1041/2079] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:32.472969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec554a7-e5fc-44b3-bedf-46c64ad8be1f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.481817 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec554a7-e5fc-44b3-bedf-46c64ad8be1f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.486889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.487538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.487815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.488092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.488814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:32.489117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.490738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.506754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1042/2080] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:14:32.507661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.508698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1039/2081] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:32 2022] PUT /identity/v3/projects/3db6db3913f64c9f8a1b0aba7bd9524f/users/f16805a7a6994701aaac66eacb94b6c6/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:32.519782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d06290-20d6-448c-a806-e49f48b6b5bf admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.523792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.524310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.524808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.566532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] MFA Rules not processed for user `f16805a7a6994701aaac66eacb94b6c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:32.568066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.584784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.585639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.585991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.586338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.587277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.589566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.671207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1040/2082] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:32.679204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.680558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.693113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1043/2083] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:32 2022] POST /identity/v3/projects => generated 367 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:32.697434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ef086-f497-40c0-8cb1-b58ae597130b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.699516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.747891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:32.774030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b8246-2d4a-4c48-8d3c-cb9ba6e0b997 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.778745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.779491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.779803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.780077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.780244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.780821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.782545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.782545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.782545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] PATH_INFO: `/v3/users/10928ccb195146a48fd71a4b4dda74aa` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.783714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.784064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10928ccb195146a48fd71a4b4dda74aa)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:32.786158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:32.874069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1044/2084] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:32 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:32.874069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:32.874069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:32.934582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5aa29a-fa71-411f-94e5-bda4682d2880 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:32.938482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:32.944813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:32.944813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:32.944813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:32.964340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:32.977596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10928ccb195146a48fd71a4b4dda74aa'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.984453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10928ccb195146a48fd71a4b4dda74aa'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.985177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10928ccb195146a48fd71a4b4dda74aa'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.996707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10928ccb195146a48fd71a4b4dda74aa'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:32.998625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.000091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:33.009295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1045/2085] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:33.010171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:33.021568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d7817-0e3c-458e-a43a-6676583d4cf5 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:33.039261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:33.039261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.039261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.039261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.039805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.039805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:33.041838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1046/2086] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:33.045640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:33.056302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c5369-838a-48e3-966d-ac7d41f45191 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:33.062221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:33.062907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.063225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.063503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.068573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.068573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:33.071269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-67dc679e-71ef-4c75-bf83-234df02f5617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10928ccb195146a48fd71a4b4dda74aa'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:33.080218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1047/2087] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:33.080443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1041/2088] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:32 2022] DELETE /identity/v3/users/10928ccb195146a48fd71a4b4dda74aa => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:33.083114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:33.083762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.084055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.084397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.085368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:33.098603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b2423-9ec2-438d-bdf9-4909b92a9ca7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:33.104441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:33.105767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:33.177571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a55365-7d74-4d8b-a195-d69a8277e6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:33.178585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a55365-7d74-4d8b-a195-d69a8277e6be admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.178928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a55365-7d74-4d8b-a195-d69a8277e6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.179276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a55365-7d74-4d8b-a195-d69a8277e6be admin admin] PATH_INFO: `/v3/projects/9e23faa208f94945bbb03eb9b562897a/users/5f250eae66c941e997153f83b329b48b/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.191141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1042/2089] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:33.197042 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a55365-7d74-4d8b-a195-d69a8277e6be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e23faa208f94945bbb03eb9b562897a, user_id=5f250eae66c941e997153f83b329b48b, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.201095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a55365-7d74-4d8b-a195-d69a8277e6be admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:33.209011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:33.213562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:33.267819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1048/2090] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:33 2022] PUT /identity/v3/projects/9e23faa208f94945bbb03eb9b562897a/users/5f250eae66c941e997153f83b329b48b/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:33.275244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3136df-2933-4f0d-a9bd-cb4574bf8a80 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:33.277954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.278246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.278524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.306478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] MFA Rules not processed for user `5f250eae66c941e997153f83b329b48b`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:33.307644 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:33.324359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:33.325012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.325319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.325599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.381531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1049/2091] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:33.392967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.394496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:33.404662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1043/2092] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:33.507885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:33.509292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:33.567423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9153c-33bf-4b27-b919-67b328c70169 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:33.571089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:33.571762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.572045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.572323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.580704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.582232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:33.591248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1050/2093] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:33.729549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:33.731712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:33.747512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c550e-7cf2-4fc6-a0c3-ddcb5cfa791e service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:33.753254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:33.754263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.754613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.754898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] PATH_INFO: `/v3/users/ccc2ed98e4df4615a380382d9b340bda` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.756747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc2ed98e4df4615a380382d9b340bda)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.758997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:33.949592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:33.951706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:33.980477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91279ae4-3869-4857-831a-62983bb0f3dd service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:33.985146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:33.986045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:33.986321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:33.986591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] PATH_INFO: `/v3/projects/3efe80a1ba96448d9bd9fce671bd183a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:33.987775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3efe80a1ba96448d9bd9fce671bd183a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:33.989713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:34.009398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc2ed98e4df4615a380382d9b340bda'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.027689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc2ed98e4df4615a380382d9b340bda'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.028477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc2ed98e4df4615a380382d9b340bda'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.044768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc2ed98e4df4615a380382d9b340bda'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.094626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3efe80a1ba96448d9bd9fce671bd183a', 'request_id': 'req-030635bb-dd6b-4770-a923-d6cebb4cec87'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.162783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-680cabbe-bb85-4bd2-b2ae-4ff1afd0279c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc2ed98e4df4615a380382d9b340bda'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.173237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1044/2094] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:33 2022] DELETE /identity/v3/users/ccc2ed98e4df4615a380382d9b340bda => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:34.178999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:34.179290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:34.179569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:34.203183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:34.205296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:34.268396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1045/2095] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:34.279148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:34.280621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:34.286584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3efe80a1ba96448d9bd9fce671bd183a', 'request_id': 'req-030635bb-dd6b-4770-a923-d6cebb4cec87'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.293607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3efe80a1ba96448d9bd9fce671bd183a', 'request_id': 'req-030635bb-dd6b-4770-a923-d6cebb4cec87'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.317172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] The token cache is being invalidate because project 3efe80a1ba96448d9bd9fce671bd183a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:34.317493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.318654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.320459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1051/2096] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:33 2022] DELETE /identity/v3/projects/3efe80a1ba96448d9bd9fce671bd183a => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:34.324309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:34.325695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:34.345359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499a5fd9-f151-4adc-a584-37bd096ea81d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:34.349685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:34.350413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:34.350898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:34.351356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:34.353347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:34.366475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:34.367933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:34.381455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1046/2097] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:34.403101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030635bb-dd6b-4770-a923-d6cebb4cec87 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:34.408587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:34.409529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:34.409818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:34.410091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] PATH_INFO: `/v3/users/b3453ce50bcd4b969a740473aae89513` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:34.411454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3453ce50bcd4b969a740473aae89513)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:34.413636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:34.540156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:34.683745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3453ce50bcd4b969a740473aae89513'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.687434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3453ce50bcd4b969a740473aae89513'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.699074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3453ce50bcd4b969a740473aae89513'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.699074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3453ce50bcd4b969a740473aae89513'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.711368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3453ce50bcd4b969a740473aae89513'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:34.784178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1052/2098] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:34 2022] DELETE /identity/v3/users/b3453ce50bcd4b969a740473aae89513 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:34.816386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:34.817867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:34.831307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877bd953-fe49-4cbb-9717-73a5fb0e5e21 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:34.893381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:34.894267 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:34.894570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:34.894846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] PATH_INFO: `/v3/projects/7df9f3e832c0492682716b721e7dfde3` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:34.896031 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df9f3e832c0492682716b721e7dfde3)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:34.897986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:34.997072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:34.998953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:35.024166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3246a0-b7f0-4e87-a100-5b1dae7f2c55 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:35.054439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df9f3e832c0492682716b721e7dfde3', 'request_id': 'req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.114473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:35.115367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:35.115653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:35.115930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] PATH_INFO: `/v3/users/92797baeea7c4a8ea56d72c030b48406` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:35.117143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=92797baeea7c4a8ea56d72c030b48406)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:35.119130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:35.179589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df9f3e832c0492682716b721e7dfde3', 'request_id': 'req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.186729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df9f3e832c0492682716b721e7dfde3', 'request_id': 'req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.197459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] The token cache is being invalidate because project 7df9f3e832c0492682716b721e7dfde3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:35.197749 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.198398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.199992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1047/2099] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:34 2022] DELETE /identity/v3/projects/7df9f3e832c0492682716b721e7dfde3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:35.204370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:35.205733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:35.208624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:35.221724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903c8ca-8435-48f3-8caa-b33c7aaf1ef4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:35.226379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:35.227255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:35.227536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:35.227864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] PATH_INFO: `/v3/users/88400114bf0642f48e7bdd8ecbd83761` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:35.229048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=88400114bf0642f48e7bdd8ecbd83761)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:35.231017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:35.417493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92797baeea7c4a8ea56d72c030b48406'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.418167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92797baeea7c4a8ea56d72c030b48406'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.425800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92797baeea7c4a8ea56d72c030b48406'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.426451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92797baeea7c4a8ea56d72c030b48406'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.438267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92797baeea7c4a8ea56d72c030b48406'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.492684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88400114bf0642f48e7bdd8ecbd83761'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.501814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88400114bf0642f48e7bdd8ecbd83761'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.502793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88400114bf0642f48e7bdd8ecbd83761'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.514725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1053/2100] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:34 2022] DELETE /identity/v3/users/92797baeea7c4a8ea56d72c030b48406 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:35.514725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:35.514725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:35.519981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88400114bf0642f48e7bdd8ecbd83761'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.582568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a9fb2-e314-4ccf-a1d4-f1e02aadd10a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:35.588087 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:35.589003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:35.589297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:35.589700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] PATH_INFO: `/v3/projects/697e3e78c05749c2b1e92d6f0affac43` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:35.590917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=697e3e78c05749c2b1e92d6f0affac43)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:35.592809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:35.606435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8640e0b0-c948-47e1-8a42-75b6546061fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88400114bf0642f48e7bdd8ecbd83761'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.626927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1048/2101] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:35 2022] DELETE /identity/v3/users/88400114bf0642f48e7bdd8ecbd83761 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:35.628812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:35.629106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:35.629427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:35.656176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:35.658228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:35.735741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1049/2102] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:35.748489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:35.750172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:35.785953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '697e3e78c05749c2b1e92d6f0affac43', 'request_id': 'req-a899964b-a076-49d6-8e2e-a5c482dcaba4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.814016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387d131-f542-4127-8951-c74f19c6d685 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:35.861593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:35.862258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:35.862543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:35.862815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:35.876503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '697e3e78c05749c2b1e92d6f0affac43', 'request_id': 'req-a899964b-a076-49d6-8e2e-a5c482dcaba4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.887862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '697e3e78c05749c2b1e92d6f0affac43', 'request_id': 'req-a899964b-a076-49d6-8e2e-a5c482dcaba4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.924996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] The token cache is being invalidate because project 697e3e78c05749c2b1e92d6f0affac43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:35.925365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.926686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:35.930734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1054/2103] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:35 2022] DELETE /identity/v3/projects/697e3e78c05749c2b1e92d6f0affac43 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:35.931934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:35.934863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:35.938133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:35.940021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:35.950157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1050/2104] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:35.950795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a899964b-a076-49d6-8e2e-a5c482dcaba4 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:35.967321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:35.967321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:35.967321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:35.967321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] PATH_INFO: `/v3/users/e489c7c99fd3443c93b48595ab35acbd` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:35.967610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e489c7c99fd3443c93b48595ab35acbd)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:35.967610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:36.218871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:36.222468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:36.235488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9362fc0b-738c-45de-9958-6886cfa0bb78 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:36.240164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:36.241241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:36.241746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:36.242621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] PATH_INFO: `/v3/projects/53ad9d224c3941079ad98dad3a7cb531` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:36.244002 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ad9d224c3941079ad98dad3a7cb531)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:36.246137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:36.471131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e489c7c99fd3443c93b48595ab35acbd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.471861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e489c7c99fd3443c93b48595ab35acbd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.476178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ad9d224c3941079ad98dad3a7cb531', 'request_id': 'req-1447b735-5796-47ed-9834-eb4597cd4170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.479980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e489c7c99fd3443c93b48595ab35acbd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.480734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e489c7c99fd3443c93b48595ab35acbd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.492768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-41f9a289-89f9-49ad-93dd-71377ff0b605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e489c7c99fd3443c93b48595ab35acbd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.562955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ad9d224c3941079ad98dad3a7cb531', 'request_id': 'req-1447b735-5796-47ed-9834-eb4597cd4170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.570738 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ad9d224c3941079ad98dad3a7cb531', 'request_id': 'req-1447b735-5796-47ed-9834-eb4597cd4170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.590240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] The token cache is being invalidate because project 53ad9d224c3941079ad98dad3a7cb531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:36.590539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.591216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.592810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1051/2105] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:36 2022] DELETE /identity/v3/projects/53ad9d224c3941079ad98dad3a7cb531 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:36.593587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:36.594986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:36.665291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b735-5796-47ed-9834-eb4597cd4170 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:36.669955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:36.670770 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:36.671057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:36.671331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] PATH_INFO: `/v3/projects/def22420c668481d8bd204e1a6056ceb` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:36.673445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=def22420c668481d8bd204e1a6056ceb)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:36.675398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:36.726913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1055/2106] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:35 2022] DELETE /identity/v3/users/e489c7c99fd3443c93b48595ab35acbd => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:36.729145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26dee-de3c-46ca-a514-4acb74f0f7aa None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:36.729796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26dee-de3c-46ca-a514-4acb74f0f7aa None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:36.730715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c26dee-de3c-46ca-a514-4acb74f0f7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:36.762387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-14c26dee-de3c-46ca-a514-4acb74f0f7aa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:36.763789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-14c26dee-de3c-46ca-a514-4acb74f0f7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:36.827845 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'def22420c668481d8bd204e1a6056ceb', 'request_id': 'req-3e75d854-698f-44e0-a73e-280b8c3320fc'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.831961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:36.838108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1056/2107] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:36.845851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ced367-1965-40be-9a3f-0f4d15db9305 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:36.846144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ced367-1965-40be-9a3f-0f4d15db9305 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:36.846453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ced367-1965-40be-9a3f-0f4d15db9305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:36.864690 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e1ced367-1965-40be-9a3f-0f4d15db9305 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:36.865909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e1ced367-1965-40be-9a3f-0f4d15db9305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:36.903954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'def22420c668481d8bd204e1a6056ceb', 'request_id': 'req-3e75d854-698f-44e0-a73e-280b8c3320fc'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.916499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'def22420c668481d8bd204e1a6056ceb', 'request_id': 'req-3e75d854-698f-44e0-a73e-280b8c3320fc'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.936241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1057/2108] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:36.936472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] The token cache is being invalidate because project def22420c668481d8bd204e1a6056ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:36.936472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.936995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ced367-1965-40be-9a3f-0f4d15db9305 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:36.937173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:36.943838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1052/2109] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:36 2022] DELETE /identity/v3/projects/def22420c668481d8bd204e1a6056ceb => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:36.945516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:36.947623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ced367-1965-40be-9a3f-0f4d15db9305 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:36.947792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:36.954883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:36.956932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:36.957328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:36.957709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:36.958566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:36.958995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:36.960887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:36.977188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1058/2110] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:36.978530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:36.981104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:36.983388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e75d854-698f-44e0-a73e-280b8c3320fc admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:36.999843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:37.002435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.002826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.003328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] PATH_INFO: `/v3/users/982dadd83d294b75a48f9f06172e8887` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.005860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] RBAC: Authorizing `identity:delete_user(user_id=982dadd83d294b75a48f9f06172e8887)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:37.009795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:37.024270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3b42ca-a657-45f4-9f0f-7c9d774de025 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:37.034929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:37.036166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.036737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.037500 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.038702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:37.042010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:37.183260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1059/2111] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:36 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:37.184363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:37.252170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbac3c62-995d-4aa1-a8f0-66af606a1ba3 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:37.257078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '982dadd83d294b75a48f9f06172e8887'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.258585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:37.259347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.259708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.260097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.260927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:37.263247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:37.267705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '982dadd83d294b75a48f9f06172e8887'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.268602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '982dadd83d294b75a48f9f06172e8887'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.274849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:37.286085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '982dadd83d294b75a48f9f06172e8887'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.448214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '982dadd83d294b75a48f9f06172e8887'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.455050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1053/2112] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:36 2022] DELETE /identity/v3/users/982dadd83d294b75a48f9f06172e8887 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:37.482321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1060/2113] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:37 2022] POST /identity/v3/users => generated 287 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:37.484679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:37.485564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:37.487052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:37.487540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:37.509784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a3ba8-6617-490c-b43e-3b243789ab25 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:37.603428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d3708-81fd-44cb-abdb-d0747a82589f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:37.608902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:37.609832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.610657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.611224 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.611754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:37.613592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:37.616565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:37.617764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.618266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.620841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] PATH_INFO: `/v3/projects/0212bfb958b04ff4b9e424e22248fdbc` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.620841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0212bfb958b04ff4b9e424e22248fdbc)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:37.622675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:37.625785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1054/2114] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:37.629839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:37.643229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e14b9-813c-4377-9275-caa0e3154a8d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:37.650405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:37.651567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.651567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.657619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.657619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:37.657619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:37.668876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1055/2115] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:37.670082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:37.682235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a0626-285a-4fb3-b15c-3b1443f1ba18 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:37.690039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611ac39-864f-4f87-a528-70e54ff6e3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:37.690943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ac39-864f-4f87-a528-70e54ff6e3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.691225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ac39-864f-4f87-a528-70e54ff6e3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.691591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611ac39-864f-4f87-a528-70e54ff6e3ed admin admin] PATH_INFO: `/v3/projects/3696faa5dcc1412b95cfdeb2fd725908/users/22bba4ba17bc4119ba03df7ea1ced48e/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.708502 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611ac39-864f-4f87-a528-70e54ff6e3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=3696faa5dcc1412b95cfdeb2fd725908, user_id=22bba4ba17bc4119ba03df7ea1ced48e, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:37.712438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611ac39-864f-4f87-a528-70e54ff6e3ed admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:37.859768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1056/2116] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:37 2022] PUT /identity/v3/projects/3696faa5dcc1412b95cfdeb2fd725908/users/22bba4ba17bc4119ba03df7ea1ced48e/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:37.871659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d083986-3312-49c4-b5e4-a84605741b03 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:37.871884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d083986-3312-49c4-b5e4-a84605741b03 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:37.872178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d083986-3312-49c4-b5e4-a84605741b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:37.877297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0212bfb958b04ff4b9e424e22248fdbc', 'request_id': 'req-f250a5f5-a876-4327-b50e-718daa5b9ac2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.912144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6d083986-3312-49c4-b5e4-a84605741b03 None None] MFA Rules not processed for user `22bba4ba17bc4119ba03df7ea1ced48e`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:37.913381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6d083986-3312-49c4-b5e4-a84605741b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:37.973782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0212bfb958b04ff4b9e424e22248fdbc', 'request_id': 'req-f250a5f5-a876-4327-b50e-718daa5b9ac2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.981577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0212bfb958b04ff4b9e424e22248fdbc', 'request_id': 'req-f250a5f5-a876-4327-b50e-718daa5b9ac2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.996330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] The token cache is being invalidate because project 0212bfb958b04ff4b9e424e22248fdbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:37.996628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:37.998173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f250a5f5-a876-4327-b50e-718daa5b9ac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.005315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1061/2117] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:37 2022] DELETE /identity/v3/projects/0212bfb958b04ff4b9e424e22248fdbc => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:38.010336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.010510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.014489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.022111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1057/2118] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:38.026470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d083986-3312-49c4-b5e4-a84605741b03 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.030913 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:38.033257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.107977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d083986-3312-49c4-b5e4-a84605741b03 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:38.109315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1062/2119] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:38.110116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.114346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.117007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608525c-0349-4bea-8474-a7adb20e8a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:38.117733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608525c-0349-4bea-8474-a7adb20e8a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.118020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608525c-0349-4bea-8474-a7adb20e8a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.118304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608525c-0349-4bea-8474-a7adb20e8a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.119013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608525c-0349-4bea-8474-a7adb20e8a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:38.121715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608525c-0349-4bea-8474-a7adb20e8a48 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:38.132030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a46660-44d8-4e52-8cb1-35003a97e7a1 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:38.137467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:38.139065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.139412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.140514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] PATH_INFO: `/v3/users/f16805a7a6994701aaac66eacb94b6c6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.142352 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16805a7a6994701aaac66eacb94b6c6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:38.144837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:38.225767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1058/2120] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:38.227225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.227520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.227799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.246118 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:38.248147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.311730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1059/2121] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:38.312683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.316130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.330556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e6c8b6-7ae9-430e-b593-6aefb4fd4c6f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:38.337557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:38.338711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.339057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.339593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] PATH_INFO: `/v3/projects/1c35f8a71d2f4820b25a3ad321d5f71c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.341045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c35f8a71d2f4820b25a3ad321d5f71c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:38.343436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:38.346139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16805a7a6994701aaac66eacb94b6c6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.347330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16805a7a6994701aaac66eacb94b6c6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.356824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16805a7a6994701aaac66eacb94b6c6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.358111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16805a7a6994701aaac66eacb94b6c6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.374443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16805a7a6994701aaac66eacb94b6c6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.486443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c35f8a71d2f4820b25a3ad321d5f71c', 'request_id': 'req-e0d72ecd-de23-4103-acbe-e5d44214eb27'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.509729 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1063/2122] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:38 2022] DELETE /identity/v3/users/f16805a7a6994701aaac66eacb94b6c6 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:38.511745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.513164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.582491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af18d4-7cef-4e37-8c33-53daed8ae7d0 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:38.587292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:38.587915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.588196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.588475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.589172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:38.591121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:38.629765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c35f8a71d2f4820b25a3ad321d5f71c', 'request_id': 'req-e0d72ecd-de23-4103-acbe-e5d44214eb27'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.637947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c35f8a71d2f4820b25a3ad321d5f71c', 'request_id': 'req-e0d72ecd-de23-4103-acbe-e5d44214eb27'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.655798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] The token cache is being invalidate because project 1c35f8a71d2f4820b25a3ad321d5f71c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:38.656138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.656943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e0d72ecd-de23-4103-acbe-e5d44214eb27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.662332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1060/2123] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:38 2022] DELETE /identity/v3/projects/1c35f8a71d2f4820b25a3ad321d5f71c => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:38.668181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f810d-ae9a-4ff1-a102-dfd91b0238d0 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.668181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f810d-ae9a-4ff1-a102-dfd91b0238d0 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.668181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f810d-ae9a-4ff1-a102-dfd91b0238d0 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.668181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1061/2124] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:14:38.668181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f810d-ae9a-4ff1-a102-dfd91b0238d0 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.669663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b03f810d-ae9a-4ff1-a102-dfd91b0238d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.686328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f810d-ae9a-4ff1-a102-dfd91b0238d0 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:38.691407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1716aa-7355-4f82-b2bb-257249c478ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:38.692139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1716aa-7355-4f82-b2bb-257249c478ef admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.692345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1716aa-7355-4f82-b2bb-257249c478ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.692673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1716aa-7355-4f82-b2bb-257249c478ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.693465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1716aa-7355-4f82-b2bb-257249c478ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:38.693773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1716aa-7355-4f82-b2bb-257249c478ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:38.695372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1716aa-7355-4f82-b2bb-257249c478ef admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:38.708330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1062/2125] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:38.709804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ce79b-7a9a-4f63-bde2-eedaf7351618 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.710132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ce79b-7a9a-4f63-bde2-eedaf7351618 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.710413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ce79b-7a9a-4f63-bde2-eedaf7351618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.729672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1064/2126] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:38.732995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-313ce79b-7a9a-4f63-bde2-eedaf7351618 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:38.733105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.733195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-313ce79b-7a9a-4f63-bde2-eedaf7351618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.733302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.740964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1063/2127] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:38.742290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f40764-86cb-4fd8-9f25-032d1b95bc59 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.742578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f40764-86cb-4fd8-9f25-032d1b95bc59 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.742859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f40764-86cb-4fd8-9f25-032d1b95bc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.744272 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2ded06-ea66-4b55-bc90-fb1106a2dc27 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:38.749961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:38.752831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.753124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.753519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] PATH_INFO: `/v3/users/2241e7dcf7b44037b1b80e0f99c8e5e8` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.755748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2241e7dcf7b44037b1b80e0f99c8e5e8)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:38.758458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:38.763006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-55f40764-86cb-4fd8-9f25-032d1b95bc59 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:38.764215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-55f40764-86cb-4fd8-9f25-032d1b95bc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.779774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1064/2128] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:38.779774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33dfa9-b2c2-40d2-9c14-ea757c741f81 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.779774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33dfa9-b2c2-40d2-9c14-ea757c741f81 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.779774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d33dfa9-b2c2-40d2-9c14-ea757c741f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.799042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-0d33dfa9-b2c2-40d2-9c14-ea757c741f81 None None] MFA Rules not processed for user `4989f980a8804f18b7ea2fec0ecad8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:38.800208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0d33dfa9-b2c2-40d2-9c14-ea757c741f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.862117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1065/2129] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:38.862867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33dfa9-b2c2-40d2-9c14-ea757c741f81 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.873364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d33dfa9-b2c2-40d2-9c14-ea757c741f81 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:38.878916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:38.879582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:38.879927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:38.880157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:38.880850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:38.882832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:38.981066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1066/2130] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:38 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:38.981066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:38.981066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:38.988024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2241e7dcf7b44037b1b80e0f99c8e5e8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.988695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2241e7dcf7b44037b1b80e0f99c8e5e8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.995891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2241e7dcf7b44037b1b80e0f99c8e5e8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:38.996531 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2241e7dcf7b44037b1b80e0f99c8e5e8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:39.009912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4ed57bdd-5789-4271-a5b1-f0da23cfd40b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2241e7dcf7b44037b1b80e0f99c8e5e8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:39.031877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:INSERT=1,keystone:DELETE=6,keystone:SELECT=93 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:39.049426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b9e747-33e7-4f7d-a9bb-d38f14062a67 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.054432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.055098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.055379 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.055654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.056419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.058052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.068751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1067/2131] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:39.069771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.071726 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.082291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1065/2132] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:38 2022] DELETE /identity/v3/users/2241e7dcf7b44037b1b80e0f99c8e5e8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:39.088248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.088248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.088248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.105114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:39.106501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.134079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65334ff9-b2eb-4c5c-bab9-4111183cf46a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.183217 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1066/2133] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:39.184439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.190152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.230773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.232549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.232911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.233258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.236423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.252954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.255288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.270550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1068/2134] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:39.271488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.273550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.276185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecfa9e-c1e2-449a-81ea-2531e9883962 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.282285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.283129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.283764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.284222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.286733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.287899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4346b5-9967-4540-887f-debcacc307ea service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.293184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.294545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.295080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.295266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.295963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.299878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.301936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.303534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.316094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1067/2135] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:39.317319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.318861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.332660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb1b009-900f-4d58-861e-0901a3db30a8 service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.345047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.346342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.346808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.347265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] PATH_INFO: `/v3/projects/24c513df53f64292b8d9fd85a156bdb7/users/a1b43316294d4a53b5b44e7ddf12506c/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.376949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c513df53f64292b8d9fd85a156bdb7, user_id=a1b43316294d4a53b5b44e7ddf12506c, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.380017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.460592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1069/2136] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:39 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:39.461611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.472084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c39353-8fd7-4a51-b94e-85c57980fb4a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.478829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.479750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.483806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.483806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.483806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:39.483806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.483806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.498459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1070/2137] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:39.499465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.513834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1068/2138] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:39 2022] PUT /identity/v3/projects/24c513df53f64292b8d9fd85a156bdb7/users/a1b43316294d4a53b5b44e7ddf12506c/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:39.513834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.520290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4ac4a7-087e-4018-937d-50843d95e018 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.602730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:39.607062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.607846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.608305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.608684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.609755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.612723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.634291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1071/2139] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:39.634291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.637341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c54e018-7914-4361-8d92-229ac314f6ae admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.638355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.644775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.645481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.645767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.646046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.646748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.649124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.663691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad77e9e2-a706-451c-a4b0-94f4b88c2faf admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.673754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.673754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.673754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.673754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] PATH_INFO: `/v3/projects/4d47f8efcc7a438c90f33178422cc6db` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.674114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d47f8efcc7a438c90f33178422cc6db)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.676250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.781413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1069/2140] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:39 2022] POST /identity/v3/projects => generated 377 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:39.783080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.838779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d47f8efcc7a438c90f33178422cc6db', 'request_id': 'req-56a42889-b044-4e21-9dd1-7699bc286bfc'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:39.852401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c412f42-435b-4fc2-a02b-7a1cec834ece admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.930695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.931494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.931852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.932200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.933074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.935116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.945331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d47f8efcc7a438c90f33178422cc6db', 'request_id': 'req-56a42889-b044-4e21-9dd1-7699bc286bfc'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:39.946193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1070/2141] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:39.947040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.952946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d47f8efcc7a438c90f33178422cc6db', 'request_id': 'req-56a42889-b044-4e21-9dd1-7699bc286bfc'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:39.961429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02d606-6db0-4d3b-971d-268d4ed878ff admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:39.966055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] The token cache is being invalidate because project 4d47f8efcc7a438c90f33178422cc6db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:39.966501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:39.966935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:39.967430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:39.967837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:39.968189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:39.968542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:39.969574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:39.970235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1072/2142] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:39 2022] DELETE /identity/v3/projects/4d47f8efcc7a438c90f33178422cc6db => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:39.971276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.971684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:39.973038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.983395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1071/2143] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:39.988589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:39.990562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:39.994655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a42889-b044-4e21-9dd1-7699bc286bfc admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.006385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89403db2-6f47-4d44-ab3c-f2110a999af1 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.011282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.011638 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.012158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.012530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.012815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] PATH_INFO: `/v3/projects/3db6db3913f64c9f8a1b0aba7bd9524f` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.012944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.013236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.013824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.014170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db6db3913f64c9f8a1b0aba7bd9524f)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.014532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.016296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.016665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.106322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1073/2144] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:39 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:40.106322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.107595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.120477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6906d7-6a78-4e07-8119-c2a6ad7fe468 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.126494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.128003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.128283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.128560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] PATH_INFO: `/v3/projects/24c513df53f64292b8d9fd85a156bdb7/users/a1b43316294d4a53b5b44e7ddf12506c/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.130713 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24c513df53f64292b8d9fd85a156bdb7, user_id=a1b43316294d4a53b5b44e7ddf12506c, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.133422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.148660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db6db3913f64c9f8a1b0aba7bd9524f', 'request_id': 'req-eb088ba8-5c14-450d-8649-d4e20ea2d97a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.160007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1074/2145] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:40 2022] PUT /identity/v3/projects/24c513df53f64292b8d9fd85a156bdb7/users/a1b43316294d4a53b5b44e7ddf12506c/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:40.160007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.160007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.217850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db6db3913f64c9f8a1b0aba7bd9524f', 'request_id': 'req-eb088ba8-5c14-450d-8649-d4e20ea2d97a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.225612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db6db3913f64c9f8a1b0aba7bd9524f', 'request_id': 'req-eb088ba8-5c14-450d-8649-d4e20ea2d97a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.226605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbf6a3-6fed-4edc-9efc-74cb9d47d7d7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.231509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.232530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.232819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.233094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] PATH_INFO: `/v3/users/b686b7d7240a484db08e2fa15380ed9c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.235646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b686b7d7240a484db08e2fa15380ed9c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.237675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.238479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] The token cache is being invalidate because project 3db6db3913f64c9f8a1b0aba7bd9524f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:40.238771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.239441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.241091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1072/2146] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:39 2022] DELETE /identity/v3/projects/3db6db3913f64c9f8a1b0aba7bd9524f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:40.242113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.245482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.315887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb088ba8-5c14-450d-8649-d4e20ea2d97a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.320799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.321740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.322027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.322305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] PATH_INFO: `/v3/projects/a805d83a8e7747528285d543eb274ab6/users/6e8f1cb9760f41c6b7c1eb379daf1b64/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.337023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a805d83a8e7747528285d543eb274ab6, user_id=6e8f1cb9760f41c6b7c1eb379daf1b64, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.339654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.437142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1073/2147] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:40 2022] PUT /identity/v3/projects/a805d83a8e7747528285d543eb274ab6/users/6e8f1cb9760f41c6b7c1eb379daf1b64/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:40.438068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.439668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.502731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b686b7d7240a484db08e2fa15380ed9c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.507243 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9c55d-a028-4c4a-91c9-ddb301e11816 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.510571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b686b7d7240a484db08e2fa15380ed9c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.511295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b686b7d7240a484db08e2fa15380ed9c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.512503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9ad5e-6643-4ab0-a89d-b2a81081d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.513167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ad5e-6643-4ab0-a89d-b2a81081d88b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.513471 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ad5e-6643-4ab0-a89d-b2a81081d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.513815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9ad5e-6643-4ab0-a89d-b2a81081d88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.514623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ad5e-6643-4ab0-a89d-b2a81081d88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.516821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9ad5e-6643-4ab0-a89d-b2a81081d88b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.525928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b686b7d7240a484db08e2fa15380ed9c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.530246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1074/2148] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:40.530246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac21257-4720-4ff2-be98-62657f09cf6c None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.530246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac21257-4720-4ff2-be98-62657f09cf6c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.530246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac21257-4720-4ff2-be98-62657f09cf6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.556119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-bac21257-4720-4ff2-be98-62657f09cf6c None None] MFA Rules not processed for user `a1b43316294d4a53b5b44e7ddf12506c`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:40.557410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bac21257-4720-4ff2-be98-62657f09cf6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.619521 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6e6bda5-e4e4-45c2-b5b7-2080c4310e0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b686b7d7240a484db08e2fa15380ed9c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:40.626082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1075/2149] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:40.627611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.627961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.628291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.633653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1075/2150] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:40 2022] DELETE /identity/v3/users/b686b7d7240a484db08e2fa15380ed9c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:40.635093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.635403 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.635662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.654840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:40.660250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.660402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:40.660967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.725936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1076/2151] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:40.727077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.728360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.730810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1076/2152] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:40.730810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.739809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.742485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e526086e-a6b9-4073-aae8-e5b9c426c7a5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.748127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d4667-2f26-48c4-b40b-586f78e9ac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.748990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d4667-2f26-48c4-b40b-586f78e9ac6e admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.749421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d4667-2f26-48c4-b40b-586f78e9ac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.749928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d4667-2f26-48c4-b40b-586f78e9ac6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.751565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d4667-2f26-48c4-b40b-586f78e9ac6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.756204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d4667-2f26-48c4-b40b-586f78e9ac6e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.772473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1077/2153] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:40.775663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.776453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.777297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.806190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:40.807676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.822648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1078/2154] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:40.822648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.823648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.825661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c200cc-7ad8-48a3-acd9-7a3fe793d318 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.830627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.831520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.831805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.832088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.834655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.881249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29d0ca-b85e-4d2c-9667-3f55d46ec7cd None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.885211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.885886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.886169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.886450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.896345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.896551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.897872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.897999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.907520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1077/2155] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:40.907784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1079/2156] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:40.908382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.908499 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.917829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb13b3fb-462d-46c1-b165-63dc6e4f83a9 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.920092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69599752-33a9-4ddf-b7cc-f3a033751fca service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.924327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.925199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.925413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.925825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.926425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:40.926733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.929555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.933088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.934146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.934898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.935250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] PATH_INFO: `/v3/projects/fc2ab1c7b6654b72959cd6d5faabe896/users/e958d1c00e034c1281a3738c21e3e8ad/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.945580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1080/2157] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:40.948137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:40.950562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:40.958800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc2ab1c7b6654b72959cd6d5faabe896, user_id=e958d1c00e034c1281a3738c21e3e8ad, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.963532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:40.966237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b46d745-1c21-4942-9114-b5cb4dd84bb7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:40.972414 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:40.973241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:40.975964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:40.975964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] PATH_INFO: `/v3/projects/08ebcb0566e84d409d849c7535b72797` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:40.975964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ebcb0566e84d409d849c7535b72797)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:40.976960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.075462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1078/2158] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:40 2022] PUT /identity/v3/projects/fc2ab1c7b6654b72959cd6d5faabe896/users/e958d1c00e034c1281a3738c21e3e8ad/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:41.075462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.104227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ebcb0566e84d409d849c7535b72797', 'request_id': 'req-a56fb225-be30-41ab-a9d3-0cdb30891cdf'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:41.146106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85eb72-821b-40da-a68e-440965635505 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.174257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ebcb0566e84d409d849c7535b72797', 'request_id': 'req-a56fb225-be30-41ab-a9d3-0cdb30891cdf'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:41.183498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ebcb0566e84d409d849c7535b72797', 'request_id': 'req-a56fb225-be30-41ab-a9d3-0cdb30891cdf'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:41.198084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] The token cache is being invalidate because project 08ebcb0566e84d409d849c7535b72797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:41.198418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:41.199089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a56fb225-be30-41ab-a9d3-0cdb30891cdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:41.202616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1081/2159] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:40 2022] DELETE /identity/v3/projects/08ebcb0566e84d409d849c7535b72797 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:41.204547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9081b72-51dc-4435-aa71-d10b20783dfb None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.204837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9081b72-51dc-4435-aa71-d10b20783dfb None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.205178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9081b72-51dc-4435-aa71-d10b20783dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.224646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd80bd6-bf49-4760-977f-1fcae7efc075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.225708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd80bd6-bf49-4760-977f-1fcae7efc075 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.225997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd80bd6-bf49-4760-977f-1fcae7efc075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.226981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd80bd6-bf49-4760-977f-1fcae7efc075 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.226981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd80bd6-bf49-4760-977f-1fcae7efc075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.228919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd80bd6-bf49-4760-977f-1fcae7efc075 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.237015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a9081b72-51dc-4435-aa71-d10b20783dfb None None] MFA Rules not processed for user `e958d1c00e034c1281a3738c21e3e8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:41.237015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a9081b72-51dc-4435-aa71-d10b20783dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:41.306038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1082/2160] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:41.307504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.307785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.308056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.326725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:41.327897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:41.336674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1083/2161] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:41.337798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.338981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:41.394810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa01e24-953c-4915-a113-2fc628281314 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.398366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.399004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.399284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.399557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.408753 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.411840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1079/2162] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:41 2022] POST /identity/v3/projects => generated 365 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:41.411949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.412027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e20a2e-13e8-46b4-8e35-0d5103d6e424 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.412027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e20a2e-13e8-46b4-8e35-0d5103d6e424 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.412027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e20a2e-13e8-46b4-8e35-0d5103d6e424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.420391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1084/2163] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:41.421208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.423641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:41.432725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a4e20a2e-13e8-46b4-8e35-0d5103d6e424 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:41.433955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a4e20a2e-13e8-46b4-8e35-0d5103d6e424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:41.437836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef4313f-176d-4c51-8976-4b97c0040e81 service glance] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.520922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1080/2164] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:41.520922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e20a2e-13e8-46b4-8e35-0d5103d6e424 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.524451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e20a2e-13e8-46b4-8e35-0d5103d6e424 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.546064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.547259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.547596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.549031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.549874 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:41.550191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.561060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.561436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.561436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.561436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.561436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.562042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.562042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.583339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1081/2165] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:41.584372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.599343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18437499-4a61-4e7e-af6a-b0e519ee83a6 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.606061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.606698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.607121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.607406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.608158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.610508 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.644055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:41.678712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1085/2166] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:41 2022] POST /identity/v3/users => generated 283 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:41.682667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.692648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03e130-bd26-49fd-aece-a003d711c1f9 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.697461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.698270 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.698552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.698829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.699558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.701147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.712347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1086/2167] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:41.720971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.730913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f84e4-765a-446c-b8f9-981f0f77329a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.736109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.736766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.737043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.737371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.738146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.739740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.754082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1082/2168] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:41 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:41.754291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1087/2169] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:41.755756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.757351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:41.773233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfecb6-c7ff-466b-8a32-991163dd63b8 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.861143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc0328-b109-460d-8efd-a310aa1acb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.862151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc0328-b109-460d-8efd-a310aa1acb4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.862445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc0328-b109-460d-8efd-a310aa1acb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.862733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc0328-b109-460d-8efd-a310aa1acb4d admin admin] PATH_INFO: `/v3/projects/be4535132aa5456c99ac0b602e8a7d17/users/8481753694d2460b85a1089704f87640/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.862848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2809d5-571f-47f4-a925-4c53ccc75b31 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:41.869231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:41.870074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:41.875240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:41.875240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:41.875240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.875240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:41.890329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc0328-b109-460d-8efd-a310aa1acb4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be4535132aa5456c99ac0b602e8a7d17, user_id=8481753694d2460b85a1089704f87640, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:41.893494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc0328-b109-460d-8efd-a310aa1acb4d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:42.042122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1088/2170] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:41 2022] PUT /identity/v3/projects/be4535132aa5456c99ac0b602e8a7d17/users/8481753694d2460b85a1089704f87640/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:42.042323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1083/2171] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:41 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:42.052978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe268fd5-15c7-466a-8c1d-f5dfad0e0189 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.053289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe268fd5-15c7-466a-8c1d-f5dfad0e0189 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.053629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe268fd5-15c7-466a-8c1d-f5dfad0e0189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:42.056385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:42.067134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf178a2-ab3d-4251-8ec6-08eea986c3e7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:42.087528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fe268fd5-15c7-466a-8c1d-f5dfad0e0189 None None] MFA Rules not processed for user `8481753694d2460b85a1089704f87640`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:42.088451 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fe268fd5-15c7-466a-8c1d-f5dfad0e0189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:42.151837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:42.153584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.153584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.154000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:42.154804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:42.156628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:42.173658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1084/2172] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:42.174951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:42.178733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1089/2173] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:42.187528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102fb1f9-372a-41d6-bfd9-71f7c69b681c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:42.193115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:42.193848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.194137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.194536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:42.195285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:42.197506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:42.199143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:42.210406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1085/2174] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:42.214372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe268fd5-15c7-466a-8c1d-f5dfad0e0189 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:42.224801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:42.226660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:42.231195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe268fd5-15c7-466a-8c1d-f5dfad0e0189 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:42.239528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:42.240632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.240952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.241232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] PATH_INFO: `/v3/projects/d21f3fc90f984d44bd75cc3fd7ebbc09/users/8d4b191d60834b8db48549a9f13c7d97/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:42.261493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d21f3fc90f984d44bd75cc3fd7ebbc09, user_id=8d4b191d60834b8db48549a9f13c7d97, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:42.264597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:42.293132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eea8928-16e4-464e-8fb7-084dd5b107fa admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:42.297121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5669dad3-4dda-4400-9323-983a24c121e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:42.298067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669dad3-4dda-4400-9323-983a24c121e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.298355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669dad3-4dda-4400-9323-983a24c121e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.298639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5669dad3-4dda-4400-9323-983a24c121e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:42.307615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669dad3-4dda-4400-9323-983a24c121e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:42.309045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5669dad3-4dda-4400-9323-983a24c121e6 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:42.319183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1086/2175] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:42.352627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1090/2176] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:42 2022] PUT /identity/v3/projects/d21f3fc90f984d44bd75cc3fd7ebbc09/users/8d4b191d60834b8db48549a9f13c7d97/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:42.363977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dbf2c-d2e9-43f3-b3ca-385d4ed42dca None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.364284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dbf2c-d2e9-43f3-b3ca-385d4ed42dca None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.364552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dbf2c-d2e9-43f3-b3ca-385d4ed42dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:42.395086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ed0dbf2c-d2e9-43f3-b3ca-385d4ed42dca None None] MFA Rules not processed for user `8d4b191d60834b8db48549a9f13c7d97`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:42.396517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ed0dbf2c-d2e9-43f3-b3ca-385d4ed42dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:42.496852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1087/2177] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:42.511590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:42.583838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191e7e0-7aca-418a-8aba-4c5697b6294b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:42.587776 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:42.588441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.588731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.589012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:42.598282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:42.599728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:42.610021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1091/2178] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:42.982100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67000072-32b0-4a08-abab-b3ae2f84ca53 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:42.982567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67000072-32b0-4a08-abab-b3ae2f84ca53 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:42.982744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67000072-32b0-4a08-abab-b3ae2f84ca53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:43.025688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-67000072-32b0-4a08-abab-b3ae2f84ca53 None None] MFA Rules not processed for user `6e8f1cb9760f41c6b7c1eb379daf1b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:43.025823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-67000072-32b0-4a08-abab-b3ae2f84ca53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:43.097468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1088/2179] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:43.238782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:43.296385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf43e56-c608-409b-8154-00ab62983044 service glance] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:43.300388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:43.301035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:43.301429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:43.301790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:43.311005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:43.312449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:43.322037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1092/2180] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:44.837477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:45.129133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:45.129682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:45.129768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:45.133062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:45.134858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:45.150499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] MFA Rules not processed for user `6e8f1cb9760f41c6b7c1eb379daf1b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:45.152114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:45.173446 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1089/2181] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:45.226238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0f742-68e8-4ea6-a92a-1718e9825006 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:45.226238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:45.226238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:45.226238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:45.226588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] PATH_INFO: `/v3/users/22bba4ba17bc4119ba03df7ea1ced48e` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:45.227894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=22bba4ba17bc4119ba03df7ea1ced48e)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:45.229836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:45.346312 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:45.347831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:45.362520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631245c-5f65-468c-b6a3-595572f73c04 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:45.367466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:45.368357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:45.368639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:45.368939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] PATH_INFO: `/v3/users/5f250eae66c941e997153f83b329b48b` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:45.370189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f250eae66c941e997153f83b329b48b)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:45.372124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:45.541536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:45.553540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22bba4ba17bc4119ba03df7ea1ced48e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.560218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22bba4ba17bc4119ba03df7ea1ced48e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.560901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22bba4ba17bc4119ba03df7ea1ced48e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.572476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22bba4ba17bc4119ba03df7ea1ced48e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.676795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-053d0d2c-d3ad-416c-aac8-14dc08771b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22bba4ba17bc4119ba03df7ea1ced48e'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.683213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1093/2182] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:45 2022] DELETE /identity/v3/users/22bba4ba17bc4119ba03df7ea1ced48e => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:45.687618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:45.687934 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:45.688242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:45.689904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f250eae66c941e997153f83b329b48b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.690629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f250eae66c941e997153f83b329b48b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.697795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f250eae66c941e997153f83b329b48b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.698434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f250eae66c941e997153f83b329b48b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.707308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:45.708516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:45.711419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-83b8e9bf-b3fa-4793-84ce-573c31c778ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f250eae66c941e997153f83b329b48b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:45.770154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1094/2183] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:45.780794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:45.784477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:45.812398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1090/2184] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:45 2022] DELETE /identity/v3/users/5f250eae66c941e997153f83b329b48b => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:45.817622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff63cb-ade7-41ac-b549-2d7efe31fd55 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:45.817950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff63cb-ade7-41ac-b549-2d7efe31fd55 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:45.818234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff63cb-ade7-41ac-b549-2d7efe31fd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:45.836122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-54ff63cb-ade7-41ac-b549-2d7efe31fd55 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:45.837419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-54ff63cb-ade7-41ac-b549-2d7efe31fd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:45.841965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c84181-b6eb-4f1c-812c-e4e8aea89e6a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:45.927962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1091/2185] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:45.950245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:45.950585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff63cb-ade7-41ac-b549-2d7efe31fd55 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:45.951279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:45.952545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:45.953092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:45.969684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:45.972008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:45.973988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff63cb-ade7-41ac-b549-2d7efe31fd55 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:45.986177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1095/2186] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:45.987151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:45.987849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:45.988278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:45.988610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:46.011524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:46.012064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:46.023603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1092/2187] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:46.674419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:46.676289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:46.682202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:46.684194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:46.699129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ff0e3-0d36-438b-b046-357f2918613a service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:46.706567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:46.707623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:46.707936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:46.708111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] PATH_INFO: `/v3/projects/9e23faa208f94945bbb03eb9b562897a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:46.709719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e23faa208f94945bbb03eb9b562897a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:46.725072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:46.725697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e7672-0e5f-4758-a7bf-d5aab44b464a service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:46.725697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:46.725697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:46.725697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:46.726507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] PATH_INFO: `/v3/projects/3696faa5dcc1412b95cfdeb2fd725908` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:46.726507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3696faa5dcc1412b95cfdeb2fd725908)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:46.727021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:46.817435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e23faa208f94945bbb03eb9b562897a', 'request_id': 'req-a6be2dbc-551e-4380-a28b-28233b09825b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:46.878417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3696faa5dcc1412b95cfdeb2fd725908', 'request_id': 'req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:46.920592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e23faa208f94945bbb03eb9b562897a', 'request_id': 'req-a6be2dbc-551e-4380-a28b-28233b09825b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:46.928760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e23faa208f94945bbb03eb9b562897a', 'request_id': 'req-a6be2dbc-551e-4380-a28b-28233b09825b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:46.940282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] The token cache is being invalidate because project 9e23faa208f94945bbb03eb9b562897a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:46.940571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:46.941214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a6be2dbc-551e-4380-a28b-28233b09825b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:46.942806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1096/2188] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:46 2022] DELETE /identity/v3/projects/9e23faa208f94945bbb03eb9b562897a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:46.950235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df960c83-6aed-4f58-9aad-dcc936b12d8f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:46.955694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df960c83-6aed-4f58-9aad-dcc936b12d8f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:46.955694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df960c83-6aed-4f58-9aad-dcc936b12d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:46.976176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-df960c83-6aed-4f58-9aad-dcc936b12d8f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:46.977420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-df960c83-6aed-4f58-9aad-dcc936b12d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:47.049552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3696faa5dcc1412b95cfdeb2fd725908', 'request_id': 'req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.062712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1097/2189] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:47.062983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3696faa5dcc1412b95cfdeb2fd725908', 'request_id': 'req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.068878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] The token cache is being invalidate because project 3696faa5dcc1412b95cfdeb2fd725908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:47.069174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.069894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.071433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1093/2190] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:46 2022] DELETE /identity/v3/projects/3696faa5dcc1412b95cfdeb2fd725908 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:47.071684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d8874-cbc0-4704-9213-397866608992 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.071979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d8874-cbc0-4704-9213-397866608992 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.072350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8d8874-cbc0-4704-9213-397866608992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.075021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.076303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:47.089105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3798626-14f7-43c3-b2f1-82e5e45fa6eb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:47.090861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2e8d8874-cbc0-4704-9213-397866608992 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:47.092152 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2e8d8874-cbc0-4704-9213-397866608992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:47.103268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:47.104197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.104455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.104727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] PATH_INFO: `/v3/users/a1b43316294d4a53b5b44e7ddf12506c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.106490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b43316294d4a53b5b44e7ddf12506c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:47.108422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:47.117807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1098/2191] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:47.119544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d8874-cbc0-4704-9213-397866608992 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.131126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8d8874-cbc0-4704-9213-397866608992 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:47.140363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:47.141720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.142058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.143182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.144005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:47.144397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:47.146416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:47.165008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1099/2192] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:14:47.170981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.177453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff5020-8c1c-4050-b226-31ecef9a391a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:47.184677 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:47.185475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.185831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.186179 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.186947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:47.189617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:47.310252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1100/2193] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:47 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:47.311862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.361002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b43316294d4a53b5b44e7ddf12506c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.361712 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b43316294d4a53b5b44e7ddf12506c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.368540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b43316294d4a53b5b44e7ddf12506c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.369199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b43316294d4a53b5b44e7ddf12506c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.383347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b43316294d4a53b5b44e7ddf12506c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:47.389088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888d5a2-677f-410b-b078-1ef456a2a822 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:47.395009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:47.395689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.395977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.396259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.396956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:47.398979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:47.489132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1094/2194] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:47 2022] DELETE /identity/v3/users/a1b43316294d4a53b5b44e7ddf12506c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:47.534243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1101/2195] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:47 2022] POST /identity/v3/users => generated 288 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:47.535345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.536796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:47.604702 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b7c81c-a20a-4ac9-a4d0-f10d51204feb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:47.609788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:47.610634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.610932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.611237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.611997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:47.613672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:47.624841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1095/2196] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:47.629546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.640025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa95ee0f-3238-4bfb-938b-fce6702770a2 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:47.645550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fe188-f85f-4158-b907-c81e324cedd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:47.646225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe188-f85f-4158-b907-c81e324cedd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.646530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe188-f85f-4158-b907-c81e324cedd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.646809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe188-f85f-4158-b907-c81e324cedd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.647541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fe188-f85f-4158-b907-c81e324cedd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:47.649151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fe188-f85f-4158-b907-c81e324cedd4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:47.659209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1102/2197] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:47.663698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.674455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65de71-38c2-4aca-b48c-5f285ca3b563 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:47.680012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:47.680978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.681285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.681724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] PATH_INFO: `/v3/projects/c2ec636b05634843b9f8eadc5ae82b1d/users/ea54d36f1dc24c38ae7c7ef34072cd83/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.697800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ec636b05634843b9f8eadc5ae82b1d, user_id=ea54d36f1dc24c38ae7c7ef34072cd83, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:47.700521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:47.820500 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1096/2198] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:47 2022] PUT /identity/v3/projects/c2ec636b05634843b9f8eadc5ae82b1d/users/ea54d36f1dc24c38ae7c7ef34072cd83/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:47.831352 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:47.831640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:47.831942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:47.859618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] MFA Rules not processed for user `ea54d36f1dc24c38ae7c7ef34072cd83`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:47.860832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:47.929623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1103/2199] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:47.940540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.942282 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:47.968814 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:47.971153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:48.025564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43791a-14d7-46e6-8d04-c59dbae813f6 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.032246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.032881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.033173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.033491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.042241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.044279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.044400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18929e-d2e4-4aba-a15a-01c60ea91681 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.055846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.056515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.057241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.057583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] PATH_INFO: `/v3/projects/24c513df53f64292b8d9fd85a156bdb7` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.058878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=24c513df53f64292b8d9fd85a156bdb7)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.060130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1097/2200] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:48.060944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.234032 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24c513df53f64292b8d9fd85a156bdb7', 'request_id': 'req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.278508 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.289893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec632fe7-edfe-4cbf-9774-70227d9cee4e service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.350121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.351074 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.351367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.351649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] PATH_INFO: `/v3/users/ea54d36f1dc24c38ae7c7ef34072cd83` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.352962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea54d36f1dc24c38ae7c7ef34072cd83)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.355052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.368069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24c513df53f64292b8d9fd85a156bdb7', 'request_id': 'req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.377435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24c513df53f64292b8d9fd85a156bdb7', 'request_id': 'req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.387642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:48.395798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] The token cache is being invalidate because project 24c513df53f64292b8d9fd85a156bdb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:48.396177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.396920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7cc0d7dc-4732-431f-a44c-b30bb9ea4bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.398746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1104/2201] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:47 2022] DELETE /identity/v3/projects/24c513df53f64292b8d9fd85a156bdb7 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:48.405706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b48e6-a678-45ba-a7dc-e906b8cb4b47 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.406005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b48e6-a678-45ba-a7dc-e906b8cb4b47 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.406511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b48e6-a678-45ba-a7dc-e906b8cb4b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.426900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6b1b48e6-a678-45ba-a7dc-e906b8cb4b47 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:48.428131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6b1b48e6-a678-45ba-a7dc-e906b8cb4b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:48.436310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1105/2202] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:48.445961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dad2f-1bb9-462d-889f-96cb80198c1e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.446245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dad2f-1bb9-462d-889f-96cb80198c1e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.446543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9dad2f-1bb9-462d-889f-96cb80198c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.466429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-af9dad2f-1bb9-462d-889f-96cb80198c1e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:48.467625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-af9dad2f-1bb9-462d-889f-96cb80198c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:48.480929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1106/2203] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:48.485144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9dad2f-1bb9-462d-889f-96cb80198c1e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.494361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9dad2f-1bb9-462d-889f-96cb80198c1e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.499059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.499746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.500532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.500849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.501666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:48.502016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.503632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.519529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1107/2204] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:14:48.520593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.531894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851aec8-d32c-456e-aa2b-dda6dc8ca737 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.537523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.538189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.538470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.541827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.541827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.541827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.635646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea54d36f1dc24c38ae7c7ef34072cd83'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.636375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea54d36f1dc24c38ae7c7ef34072cd83'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.644526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea54d36f1dc24c38ae7c7ef34072cd83'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.649160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea54d36f1dc24c38ae7c7ef34072cd83'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.665188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-59b29c43-3218-4676-b81f-38153ad4af91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea54d36f1dc24c38ae7c7ef34072cd83'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:48.665315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1108/2205] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:48 2022] POST /identity/v3/projects => generated 373 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:48.667956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.669747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:48.737876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb5145-26be-49d6-a621-27c6b4ce6e5c admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.742850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.745202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.745202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.745202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.745202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.746926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.782631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1098/2206] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:48 2022] DELETE /identity/v3/users/ea54d36f1dc24c38ae7c7ef34072cd83 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:48.784785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.785080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.785392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.802960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:48.804532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:48.867047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1099/2207] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:48.877751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.879271 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:48.885307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1109/2208] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:48 2022] POST /identity/v3/users => generated 287 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:48.897481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.908668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de1bd0-988e-467e-b11e-c265c2a71217 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.913854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.914632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.915036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.915404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.916378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.918239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.935157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1110/2209] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:48.939828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.949831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5064cc-4faf-43c6-842c-d1c27b441fc0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.951419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e821446-ba33-4e7a-9443-a88641750ab4 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.955022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.955694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.956045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.956272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.957008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.957397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf9fec-31bb-4531-9513-5514287e237c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.958113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf9fec-31bb-4531-9513-5514287e237c service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.967302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf9fec-31bb-4531-9513-5514287e237c service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.967302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf9fec-31bb-4531-9513-5514287e237c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.967549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.969010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf9fec-31bb-4531-9513-5514287e237c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:48.969226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1111/2210] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:48.970655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf9fec-31bb-4531-9513-5514287e237c service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:48.973529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:48.981051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1100/2211] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:14:48.983967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c31c3-c183-4cd5-9885-cafecfde8615 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:48.989254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:48.990151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:48.990434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:48.990705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] PATH_INFO: `/v3/projects/348c66c9e8fd470a940a114828eed387/users/c521c0261ad149dea16383197620b0e5/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:48.995557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:49.010010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=348c66c9e8fd470a940a114828eed387, user_id=c521c0261ad149dea16383197620b0e5, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:49.011091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:49.179443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1112/2212] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:48 2022] PUT /identity/v3/projects/348c66c9e8fd470a940a114828eed387/users/c521c0261ad149dea16383197620b0e5/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:49.186820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:49.187119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:49.187405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:49.221030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] MFA Rules not processed for user `c521c0261ad149dea16383197620b0e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:49.222299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:49.296479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1101/2213] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:49.308142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:49.310077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:49.374094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d9a8c-220f-4808-97a2-d65e55a55648 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:49.378540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36897d-0e83-46d6-8541-7dafd85a61b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:49.379394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36897d-0e83-46d6-8541-7dafd85a61b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:49.379779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36897d-0e83-46d6-8541-7dafd85a61b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:49.380167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36897d-0e83-46d6-8541-7dafd85a61b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:49.391338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36897d-0e83-46d6-8541-7dafd85a61b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:49.395143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36897d-0e83-46d6-8541-7dafd85a61b4 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:49.402793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1113/2214] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:49.491856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:49.493324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:49.574974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b858122-c5b7-41eb-b75b-37fbeb3b28e3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:49.578134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:49.579368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:49.579663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:49.580600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] PATH_INFO: `/v3/projects/c2ec636b05634843b9f8eadc5ae82b1d` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:49.581934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ec636b05634843b9f8eadc5ae82b1d)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:49.584405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:49.765362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ec636b05634843b9f8eadc5ae82b1d', 'request_id': 'req-c69308b9-ae95-492c-93da-4105c89fbdb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:49.927426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ec636b05634843b9f8eadc5ae82b1d', 'request_id': 'req-c69308b9-ae95-492c-93da-4105c89fbdb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:49.934012 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ec636b05634843b9f8eadc5ae82b1d', 'request_id': 'req-c69308b9-ae95-492c-93da-4105c89fbdb9'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:49.946345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] The token cache is being invalidate because project c2ec636b05634843b9f8eadc5ae82b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:49.946717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:49.947439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c69308b9-ae95-492c-93da-4105c89fbdb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:49.949067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1102/2215] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:49 2022] DELETE /identity/v3/projects/c2ec636b05634843b9f8eadc5ae82b1d => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:49.954380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc028980-0229-45aa-b2b6-fbe69ee93347 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:49.954927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc028980-0229-45aa-b2b6-fbe69ee93347 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:49.954927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc028980-0229-45aa-b2b6-fbe69ee93347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:49.975511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-cc028980-0229-45aa-b2b6-fbe69ee93347 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:49.976812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cc028980-0229-45aa-b2b6-fbe69ee93347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:50.038528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1114/2216] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:50.048542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c20a-d594-4bab-b4da-de9d53f7f5a6 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.048834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c20a-d594-4bab-b4da-de9d53f7f5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.049110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0c20a-d594-4bab-b4da-de9d53f7f5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.067990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-63c0c20a-d594-4bab-b4da-de9d53f7f5a6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:50.069198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-63c0c20a-d594-4bab-b4da-de9d53f7f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:50.077256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1103/2217] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:50.081580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc028980-0229-45aa-b2b6-fbe69ee93347 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:50.091705 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc028980-0229-45aa-b2b6-fbe69ee93347 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:50.096869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:50.097564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.097849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.098125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.098859 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:50.099169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:50.100766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:50.112668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1115/2218] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:50.116908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0c20a-d594-4bab-b4da-de9d53f7f5a6 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:50.127105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0c20a-d594-4bab-b4da-de9d53f7f5a6 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:50.131865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:50.132517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.132815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.133102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.133889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:50.135847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:50.262225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1104/2219] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:50 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:50.266935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:50.277387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d68166-0d3f-4d1c-962c-aaefea55347a admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:50.337062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:50.338169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.338609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.338890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.339629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:50.341668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:50.486666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1116/2220] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:50 2022] POST /identity/v3/users => generated 294 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:50.491594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:50.502783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203762a-5fd6-4a32-9ce3-1f81cdb208c9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:50.507587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:50.508315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.508653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.509164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.510025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:50.511685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:50.525464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1105/2221] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:50.534458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:50.563080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b2999-8b4c-4725-848a-70ea7b072375 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:50.570473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e16f0-cbce-4c0f-8be1-b61097a872ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:50.571255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e16f0-cbce-4c0f-8be1-b61097a872ed admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.571543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e16f0-cbce-4c0f-8be1-b61097a872ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.571819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e16f0-cbce-4c0f-8be1-b61097a872ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.572557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e16f0-cbce-4c0f-8be1-b61097a872ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:50.576982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e16f0-cbce-4c0f-8be1-b61097a872ed admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:50.612313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1117/2222] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:50.613457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:50.631483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa239f-e9cb-4d90-946a-d405b459d04b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:50.636829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:50.637960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.638426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.639266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] PATH_INFO: `/v3/projects/fa921ad1ca0a4788868881d683bbe3c7/users/406916de66af4a8c82e177092143099a/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.660306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa921ad1ca0a4788868881d683bbe3c7, user_id=406916de66af4a8c82e177092143099a, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:50.663744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:50.777935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1106/2223] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:50 2022] PUT /identity/v3/projects/fa921ad1ca0a4788868881d683bbe3c7/users/406916de66af4a8c82e177092143099a/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:50.790653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f005e-180a-40c0-8734-de01b36aa3dc None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.790943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f005e-180a-40c0-8734-de01b36aa3dc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.791223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f005e-180a-40c0-8734-de01b36aa3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.824256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-797f005e-180a-40c0-8734-de01b36aa3dc None None] MFA Rules not processed for user `406916de66af4a8c82e177092143099a`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:50.825522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-797f005e-180a-40c0-8734-de01b36aa3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:50.894161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1118/2224] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:50.898605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:50.964775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa07d2-c227-4eb7-9ba0-bf810dfc2a76 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:50.970603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:50.971296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:50.971584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:50.971865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:50.972564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:50.974544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:51.123125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1107/2225] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:50 2022] POST /identity/v3/projects => generated 389 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:51.127342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f005e-180a-40c0-8734-de01b36aa3dc None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:51.140349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f005e-180a-40c0-8734-de01b36aa3dc None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:51.220615 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:51.221809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:51.222105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:51.231640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:51.231640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:51.231640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:51.323595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:51.330656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1119/2226] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:51 2022] POST /identity/v3/users => generated 301 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:51.334699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:51.345310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6aa645-36fd-4c77-90f2-f65ad9effd7d admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:51.350242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:51.350975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:51.351326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:51.351669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:51.352465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:51.354317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:51.364407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1108/2227] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:51.368656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:51.380319 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268eae31-e05a-4a64-b5eb-32436a1469f8 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:51.385328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:51.386411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:51.386707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:51.386985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] PATH_INFO: `/v3/projects/023d195e15ac491a8525cfab2ad34dfb/users/e6a964daf0c840e3841e2ee17265c769/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:51.403788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=023d195e15ac491a8525cfab2ad34dfb, user_id=e6a964daf0c840e3841e2ee17265c769, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:51.406873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:51.552922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1120/2228] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:51 2022] PUT /identity/v3/projects/023d195e15ac491a8525cfab2ad34dfb/users/e6a964daf0c840e3841e2ee17265c769/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:51.561374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:51.576900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197dece-f48e-4cde-8769-656ec9b872ea admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:51.670762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7455ce-312b-45a9-a2b7-d39325544d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:51.672437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7455ce-312b-45a9-a2b7-d39325544d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:51.672750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7455ce-312b-45a9-a2b7-d39325544d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:51.673013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7455ce-312b-45a9-a2b7-d39325544d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:51.674166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7455ce-312b-45a9-a2b7-d39325544d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:51.676108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7455ce-312b-45a9-a2b7-d39325544d63 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:51.692814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1109/2229] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:51.693906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:51.712802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc5cbf-498c-4bce-905d-7076e751d771 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:51.718199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:51.719146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:51.719792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:51.720143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] PATH_INFO: `/v3/projects/023d195e15ac491a8525cfab2ad34dfb/users/e6a964daf0c840e3841e2ee17265c769/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:51.722312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=023d195e15ac491a8525cfab2ad34dfb, user_id=e6a964daf0c840e3841e2ee17265c769, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:51.725177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:51.739074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1121/2230] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:51 2022] PUT /identity/v3/projects/023d195e15ac491a8525cfab2ad34dfb/users/e6a964daf0c840e3841e2ee17265c769/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:51.749474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeced5a8-41d1-4cd0-9f17-6b9ab5c368f9 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:51.749876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeced5a8-41d1-4cd0-9f17-6b9ab5c368f9 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:51.750830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeced5a8-41d1-4cd0-9f17-6b9ab5c368f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:51.786434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-aeced5a8-41d1-4cd0-9f17-6b9ab5c368f9 None None] MFA Rules not processed for user `e6a964daf0c840e3841e2ee17265c769`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:51.788078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-aeced5a8-41d1-4cd0-9f17-6b9ab5c368f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:51.850997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:51.880572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1110/2231] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:52.065161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:52.366383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:52.368949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:52.445354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847559a-fbe0-48c4-98e8-969d140e1eb9 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:52.449015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3b41f-d39a-4f58-8990-902c07291ee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:52.449688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3b41f-d39a-4f58-8990-902c07291ee2 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:52.449971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3b41f-d39a-4f58-8990-902c07291ee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:52.450314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3b41f-d39a-4f58-8990-902c07291ee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:52.460949 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3b41f-d39a-4f58-8990-902c07291ee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:52.462693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3b41f-d39a-4f58-8990-902c07291ee2 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:52.479445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1122/2232] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:52.692022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeced5a8-41d1-4cd0-9f17-6b9ab5c368f9 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:52.759959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeced5a8-41d1-4cd0-9f17-6b9ab5c368f9 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:52.764494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:52.765403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:52.766492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:52.766779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] PATH_INFO: `/v3/users/406916de66af4a8c82e177092143099a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:52.767963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=406916de66af4a8c82e177092143099a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:52.769926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:53.106137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406916de66af4a8c82e177092143099a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:53.106798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406916de66af4a8c82e177092143099a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:53.114278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406916de66af4a8c82e177092143099a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:53.114899 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406916de66af4a8c82e177092143099a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:53.127153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406916de66af4a8c82e177092143099a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:53.229768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1111/2233] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:52 2022] DELETE /identity/v3/users/406916de66af4a8c82e177092143099a => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:53.234999 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:53.235377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:53.235661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:53.258227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:53.259412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:53.319641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1123/2234] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:53.329576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:53.330973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:53.395630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d55c2-3ee5-42f8-8df8-308fa8b57902 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:53.401006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:53.401791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:53.402124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:53.402444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:53.413083 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:53.414555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:53.425456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1112/2235] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:53.683080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:53.924391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:53.925892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:53.949576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fa926-c8dd-4609-a049-688f3915fba6 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:53.957108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:53.959886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:53.959886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:53.959886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] PATH_INFO: `/v3/projects/fa921ad1ca0a4788868881d683bbe3c7` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:53.960445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa921ad1ca0a4788868881d683bbe3c7)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:53.962227 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:54.030606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:54.083289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa921ad1ca0a4788868881d683bbe3c7', 'request_id': 'req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.235382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa921ad1ca0a4788868881d683bbe3c7', 'request_id': 'req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.242130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa921ad1ca0a4788868881d683bbe3c7', 'request_id': 'req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.253231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] The token cache is being invalidate because project fa921ad1ca0a4788868881d683bbe3c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:54.253548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.254177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.255712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1124/2236] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:53 2022] DELETE /identity/v3/projects/fa921ad1ca0a4788868881d683bbe3c7 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:54.259725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:54.269578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b244e-b883-486a-808a-248ef181186d service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:54.329625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:54.330507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:54.330507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:54.331081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] PATH_INFO: `/v3/users/e6a964daf0c840e3841e2ee17265c769` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:54.332297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6a964daf0c840e3841e2ee17265c769)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:54.334295 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:54.604024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6a964daf0c840e3841e2ee17265c769'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.604902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6a964daf0c840e3841e2ee17265c769'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.616195 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6a964daf0c840e3841e2ee17265c769'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.617162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6a964daf0c840e3841e2ee17265c769'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.630009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-35a5b444-2808-49d2-9d39-760c5a79e41e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6a964daf0c840e3841e2ee17265c769'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:54.723078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1113/2237] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:14:54 2022] DELETE /identity/v3/users/e6a964daf0c840e3841e2ee17265c769 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:14:55.180608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:55.182137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:55.196539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df77ce8-4a7a-4871-aaa2-6be09c022fc4 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:55.254069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:55.254867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:55.255149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:55.255425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] PATH_INFO: `/v3/projects/023d195e15ac491a8525cfab2ad34dfb` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:55.256589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=023d195e15ac491a8525cfab2ad34dfb)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:55.258568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:55.365843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '023d195e15ac491a8525cfab2ad34dfb', 'request_id': 'req-6c6eff7a-4452-4458-b5e3-cae105768aa6'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:55.447399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '023d195e15ac491a8525cfab2ad34dfb', 'request_id': 'req-6c6eff7a-4452-4458-b5e3-cae105768aa6'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:55.454644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '023d195e15ac491a8525cfab2ad34dfb', 'request_id': 'req-6c6eff7a-4452-4458-b5e3-cae105768aa6'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:55.465111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] The token cache is being invalidate because project 023d195e15ac491a8525cfab2ad34dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:14:55.465441 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:55.466068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6c6eff7a-4452-4458-b5e3-cae105768aa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:14:55.467711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1125/2238] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:14:55 2022] DELETE /identity/v3/projects/023d195e15ac491a8525cfab2ad34dfb => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:55.473022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e5184-8315-4872-876c-5a88e2ee9bed None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:55.473533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e5184-8315-4872-876c-5a88e2ee9bed None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:55.473660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e5184-8315-4872-876c-5a88e2ee9bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:55.477350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:55.493908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e73e5184-8315-4872-876c-5a88e2ee9bed None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:55.495090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e73e5184-8315-4872-876c-5a88e2ee9bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:55.554937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1114/2239] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:55.564796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4967db-634f-4b40-9a04-d0353e6909e5 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:55.565089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4967db-634f-4b40-9a04-d0353e6909e5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:55.565397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4967db-634f-4b40-9a04-d0353e6909e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:55.583836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-be4967db-634f-4b40-9a04-d0353e6909e5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:55.585023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-be4967db-634f-4b40-9a04-d0353e6909e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:55.593087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1126/2240] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:55.597206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e5184-8315-4872-876c-5a88e2ee9bed None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:55.607190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e5184-8315-4872-876c-5a88e2ee9bed None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:55.612695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:55.613411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:55.613701 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:55.613990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:55.614724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:14:55.615037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:55.616767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:55.629064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:55.629721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1115/2241] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:14:55.634095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4967db-634f-4b40-9a04-d0353e6909e5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:55.643860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4967db-634f-4b40-9a04-d0353e6909e5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:55.648527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:55.649205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:55.649514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:55.649791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:55.650496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:55.652418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:55.787894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1127/2242] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:55 2022] POST /identity/v3/projects => generated 375 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:55.792289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:55.804578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ad3cd-2352-46d1-8661-3ee710ffe1ff admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:55.872976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:55.873781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:55.874129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:55.874617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:55.875550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:55.877820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:55.985594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1116/2243] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:55 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:55.989328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:55.999504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33b5330-153c-4c05-9218-d7f08bae0171 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.004074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.004732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.005156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.005255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.006039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.007643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:56.017497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1128/2244] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:56.021534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:56.032543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c704b8-fcdf-41b1-a9fe-f7ccc1ccaa9e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.037156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee5c35-a9f2-494e-8e27-ed6d11470771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.037854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee5c35-a9f2-494e-8e27-ed6d11470771 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.039543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee5c35-a9f2-494e-8e27-ed6d11470771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.039543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee5c35-a9f2-494e-8e27-ed6d11470771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.039543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee5c35-a9f2-494e-8e27-ed6d11470771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.040711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee5c35-a9f2-494e-8e27-ed6d11470771 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:56.052136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1117/2245] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:56.056290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:56.066857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580c865-cf24-4f34-8527-21a58cdf7bb1 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.071447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.072325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.072611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.072890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] PATH_INFO: `/v3/projects/dc56069dd63741cf93e73ccc76e57aa8/users/e2bbb4e02f7d46119b64ca2171bd5242/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.087475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc56069dd63741cf93e73ccc76e57aa8, user_id=e2bbb4e02f7d46119b64ca2171bd5242, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.090153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:56.179692 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1129/2246] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:56 2022] PUT /identity/v3/projects/dc56069dd63741cf93e73ccc76e57aa8/users/e2bbb4e02f7d46119b64ca2171bd5242/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:56.189907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e5e4-0a93-4f60-9fe6-3383d0285b43 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.190201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e5e4-0a93-4f60-9fe6-3383d0285b43 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.190482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1e5e4-0a93-4f60-9fe6-3383d0285b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.219297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c6e1e5e4-0a93-4f60-9fe6-3383d0285b43 None None] MFA Rules not processed for user `e2bbb4e02f7d46119b64ca2171bd5242`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:56.220703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c6e1e5e4-0a93-4f60-9fe6-3383d0285b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:56.288512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1118/2247] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:56.343144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:56.355336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f84bc5-4707-4fa5-bc41-0688b77160fe admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.360040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.360696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.360980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.361261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.361978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.363870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:56.590589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:56.672640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1130/2248] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:56 2022] POST /identity/v3/projects => generated 379 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:56.676802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1e5e4-0a93-4f60-9fe6-3383d0285b43 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:56.687820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1e5e4-0a93-4f60-9fe6-3383d0285b43 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.744538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.745233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.745594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.745874 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.746576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.748566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:56.790020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:56.850827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1119/2249] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:56 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:56.851269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:56.861717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d254628a-7d82-4336-996d-656bdce24a94 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.866560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.867242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.867525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.867802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.868532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.870204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:56.881593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1131/2250] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:56.885573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:56.896376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c160cbf-946e-401b-9e8c-37a2aa22da5b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.901296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14105a0-1152-412e-baaf-f161539c3518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.902017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14105a0-1152-412e-baaf-f161539c3518 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.902317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14105a0-1152-412e-baaf-f161539c3518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.902649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14105a0-1152-412e-baaf-f161539c3518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.903985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14105a0-1152-412e-baaf-f161539c3518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.905314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14105a0-1152-412e-baaf-f161539c3518 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:56.916332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1120/2251] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:14:56.920885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:56.931714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339c4e8-380d-45ba-b531-579c5b53ac84 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:56.936290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:56.937186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:56.938233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:56.938514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] PATH_INFO: `/v3/projects/331e3ed5019c4986abddf446c8b2ba35/users/45166c6b507b48fea9e2f3599203e9ea/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:56.952501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=331e3ed5019c4986abddf446c8b2ba35, user_id=45166c6b507b48fea9e2f3599203e9ea, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:56.955383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:57.058361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1132/2252] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:56 2022] PUT /identity/v3/projects/331e3ed5019c4986abddf446c8b2ba35/users/45166c6b507b48fea9e2f3599203e9ea/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:57.069011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c9274-9573-4daf-8585-3126b0711352 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:57.069331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c9274-9573-4daf-8585-3126b0711352 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:57.069615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c9274-9573-4daf-8585-3126b0711352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:57.096754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5a6c9274-9573-4daf-8585-3126b0711352 None None] MFA Rules not processed for user `45166c6b507b48fea9e2f3599203e9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:57.098130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5a6c9274-9573-4daf-8585-3126b0711352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:57.175546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1121/2253] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:14:57.179763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:57.245186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7c2c1-36be-4b67-9ee5-83a76b21f5f7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:57.250663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:57.251339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:57.251620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:57.251898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:57.252605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:57.254689 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:57.403368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1133/2254] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:14:57 2022] POST /identity/v3/projects => generated 377 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:14:57.407991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c9274-9573-4daf-8585-3126b0711352 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:57.418498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c9274-9573-4daf-8585-3126b0711352 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:57.477170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:57.477884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:57.478193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:57.478467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:57.479169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:57.481145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:57.642070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1122/2255] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:14:57 2022] POST /identity/v3/users => generated 295 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:14:57.645890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:57.651965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:57.656074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1273b-b326-43b6-ab15-946c779df415 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:57.662149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:57.662837 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:57.663121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:57.663393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:57.664137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:57.665815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:57.687969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1134/2256] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:57.688683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:57.695152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236c935c-ef53-461b-9f4b-21f5d6bded1c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:57.699970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:57.700725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:57.701068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:57.701376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] PATH_INFO: `/v3/projects/0c4052c220d34f5fa2cde903771a5607/users/268f0da98d8c4060a43cc4b70b239f85/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:57.719866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4052c220d34f5fa2cde903771a5607, user_id=268f0da98d8c4060a43cc4b70b239f85, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:57.723146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:57.877393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1123/2257] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:57 2022] PUT /identity/v3/projects/0c4052c220d34f5fa2cde903771a5607/users/268f0da98d8c4060a43cc4b70b239f85/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:57.879732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:57.893957 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef5a359-58ba-4c60-b734-b39292bbf9e0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:57.977927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e353d2ae-f0d7-4f6b-a109-e328fe17d963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:57.978599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d2ae-f0d7-4f6b-a109-e328fe17d963 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:57.978878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d2ae-f0d7-4f6b-a109-e328fe17d963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:57.979153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e353d2ae-f0d7-4f6b-a109-e328fe17d963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:57.979935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353d2ae-f0d7-4f6b-a109-e328fe17d963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:57.981598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e353d2ae-f0d7-4f6b-a109-e328fe17d963 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:57.994585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1135/2258] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:14:57.995822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:58.013732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b147c55-4272-4c6b-b9f9-f7a86cfab0dd admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:58.024499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:58.024499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:58.024499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:58.024499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] PATH_INFO: `/v3/projects/0c4052c220d34f5fa2cde903771a5607/users/268f0da98d8c4060a43cc4b70b239f85/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:58.025090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c4052c220d34f5fa2cde903771a5607, user_id=268f0da98d8c4060a43cc4b70b239f85, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:58.025372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:58.037666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1124/2259] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:14:57 2022] PUT /identity/v3/projects/0c4052c220d34f5fa2cde903771a5607/users/268f0da98d8c4060a43cc4b70b239f85/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:14:58.048649 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c506ce0-12f5-470f-9337-71e00603120a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:58.048960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c506ce0-12f5-470f-9337-71e00603120a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:58.049233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c506ce0-12f5-470f-9337-71e00603120a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:58.082897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8c506ce0-12f5-470f-9337-71e00603120a None None] MFA Rules not processed for user `268f0da98d8c4060a43cc4b70b239f85`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:14:58.084116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8c506ce0-12f5-470f-9337-71e00603120a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:58.162077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1136/2260] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:14:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:14:58.213293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:58.220025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:14:58.294187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:58.304663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17fc58-4bc6-484c-8ef6-d5d8c055cd60 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:58.309201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:58.309921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:58.310211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:58.310489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:58.321942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:58.323348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:58.337285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1125/2261] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:58 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:14:58.465001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:14:58.563595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c506ce0-12f5-470f-9337-71e00603120a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:14:58.574550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c506ce0-12f5-470f-9337-71e00603120a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:14:58.580418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dea530-b301-4c9d-8fdb-0fd315770ff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:14:58.581107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dea530-b301-4c9d-8fdb-0fd315770ff8 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:14:58.581432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dea530-b301-4c9d-8fdb-0fd315770ff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:14:58.581712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dea530-b301-4c9d-8fdb-0fd315770ff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:14:58.641968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dea530-b301-4c9d-8fdb-0fd315770ff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:14:58.643676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dea530-b301-4c9d-8fdb-0fd315770ff8 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:14:58.654351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1137/2262] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:14:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:00.139315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:00.646612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:01.590527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:01.881714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:02.809412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:03.659078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=85,keystone:INSERT=8,keystone:DELETE=7 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:03.996282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:04.381567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:04.457732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d203fe6-f864-464b-8d45-44b114ce253f service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:04.462626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:04.463495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:04.463779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:04.464056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] PATH_INFO: `/v3/users/e2bbb4e02f7d46119b64ca2171bd5242` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:04.465332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bbb4e02f7d46119b64ca2171bd5242)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:04.467262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:04.691264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bbb4e02f7d46119b64ca2171bd5242'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:04.691955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bbb4e02f7d46119b64ca2171bd5242'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:04.698885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bbb4e02f7d46119b64ca2171bd5242'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:04.699559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bbb4e02f7d46119b64ca2171bd5242'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:04.713484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bbb4e02f7d46119b64ca2171bd5242'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:04.802128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1126/2263] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:04 2022] DELETE /identity/v3/users/e2bbb4e02f7d46119b64ca2171bd5242 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:04.806703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:04.807001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:04.807304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:04.826236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:04.827427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:04.889219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1138/2264] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:04.899085 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:04.900495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:04.961447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa352d7-508d-4a03-8473-ed8edf2368f6 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:04.965191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:04.965940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:04.966233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:04.966568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:04.977031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:04.978618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:04.989815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1127/2265] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:05.167322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:05.500048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:05.502604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:05.526750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:05.526936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92032c44-6677-419c-aa53-eeb4bdefb9f8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:05.532231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:05.539516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:05.539516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:05.539516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] PATH_INFO: `/v3/projects/dc56069dd63741cf93e73ccc76e57aa8` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:05.539516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc56069dd63741cf93e73ccc76e57aa8)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:05.539516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:05.681291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc56069dd63741cf93e73ccc76e57aa8', 'request_id': 'req-4f5c8666-4975-4e69-a84c-c5d492d0d39d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:05.802548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc56069dd63741cf93e73ccc76e57aa8', 'request_id': 'req-4f5c8666-4975-4e69-a84c-c5d492d0d39d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:05.803475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:05.804935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:05.810494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc56069dd63741cf93e73ccc76e57aa8', 'request_id': 'req-4f5c8666-4975-4e69-a84c-c5d492d0d39d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:05.820210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae54f148-c779-4a42-ac87-87bd51f5e13c service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:05.825306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] The token cache is being invalidate because project dc56069dd63741cf93e73ccc76e57aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:05.825500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:05.826854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:05.831988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1139/2266] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:05 2022] DELETE /identity/v3/projects/dc56069dd63741cf93e73ccc76e57aa8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:05.837954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:05.839381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:05.892602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:05.893728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:05.893888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:05.894139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] PATH_INFO: `/v3/users/c521c0261ad149dea16383197620b0e5` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:05.896014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c521c0261ad149dea16383197620b0e5)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:05.897982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:05.915203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5c8666-4975-4e69-a84c-c5d492d0d39d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:05.920008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:05.920892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:05.921179 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:05.922050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] PATH_INFO: `/v3/users/45166c6b507b48fea9e2f3599203e9ea` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:05.929607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=45166c6b507b48fea9e2f3599203e9ea)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:05.929607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:06.106835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c521c0261ad149dea16383197620b0e5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.107505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c521c0261ad149dea16383197620b0e5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.115612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c521c0261ad149dea16383197620b0e5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.116298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c521c0261ad149dea16383197620b0e5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.130794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e4df8bad-847e-4b40-8224-ca68bcd9201e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c521c0261ad149dea16383197620b0e5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.165736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45166c6b507b48fea9e2f3599203e9ea'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.173797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45166c6b507b48fea9e2f3599203e9ea'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.174516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45166c6b507b48fea9e2f3599203e9ea'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.188142 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45166c6b507b48fea9e2f3599203e9ea'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.258351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1128/2267] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:05 2022] DELETE /identity/v3/users/c521c0261ad149dea16383197620b0e5 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:06.263739 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:06.264083 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:06.264362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:06.282086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:06.283284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:06.343872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1129/2268] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:06.353570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:06.354948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:06.367010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45166c6b507b48fea9e2f3599203e9ea'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:06.374110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1140/2269] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:05 2022] DELETE /identity/v3/users/45166c6b507b48fea9e2f3599203e9ea => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:06.408709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:06.414623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e76ee93-4bc4-420d-af66-09c29dd1419e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:06.536170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:06.536976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:06.537309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:06.538567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:06.541009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:06.636649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:06.638180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:06.649692 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1130/2270] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:15:06.847535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:07.175463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:07.177729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:07.216440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ead4cc-0ea1-4586-a22f-be35329bf0ac admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:07.229432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:07.230517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:07.230874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:07.231217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] PATH_INFO: `/v3/projects/331e3ed5019c4986abddf446c8b2ba35` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:07.233621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=331e3ed5019c4986abddf446c8b2ba35)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:07.236024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:07.371668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:07.373580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:07.378041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '331e3ed5019c4986abddf446c8b2ba35', 'request_id': 'req-12d673b1-97d0-470c-bfca-bb0764edc4e3'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.385770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a57002-9c70-4884-abfc-0e113a998d62 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:07.447120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '331e3ed5019c4986abddf446c8b2ba35', 'request_id': 'req-12d673b1-97d0-470c-bfca-bb0764edc4e3'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.454034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:07.457537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:07.457537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:07.457537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] PATH_INFO: `/v3/projects/348c66c9e8fd470a940a114828eed387` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:07.457801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '331e3ed5019c4986abddf446c8b2ba35', 'request_id': 'req-12d673b1-97d0-470c-bfca-bb0764edc4e3'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.457878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=348c66c9e8fd470a940a114828eed387)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:07.459797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:07.470905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] The token cache is being invalidate because project 331e3ed5019c4986abddf446c8b2ba35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:07.471731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.472871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.475065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1141/2271] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:07 2022] DELETE /identity/v3/projects/331e3ed5019c4986abddf446c8b2ba35 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:07.479034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:07.480343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:07.495355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d673b1-97d0-470c-bfca-bb0764edc4e3 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:07.503683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:07.503683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:07.503683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:07.503683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] PATH_INFO: `/v3/users/268f0da98d8c4060a43cc4b70b239f85` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:07.504149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=268f0da98d8c4060a43cc4b70b239f85)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:07.506078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:07.619697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '348c66c9e8fd470a940a114828eed387', 'request_id': 'req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.717996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:07.740762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '268f0da98d8c4060a43cc4b70b239f85'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.744975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '348c66c9e8fd470a940a114828eed387', 'request_id': 'req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.752494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '268f0da98d8c4060a43cc4b70b239f85'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.753632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '268f0da98d8c4060a43cc4b70b239f85'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.753827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '348c66c9e8fd470a940a114828eed387', 'request_id': 'req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.769875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] The token cache is being invalidate because project 348c66c9e8fd470a940a114828eed387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:07.770104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.770780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4d6afc7c-3dcc-4bf5-9e02-399ba15137d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.777038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1131/2272] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:07 2022] DELETE /identity/v3/projects/348c66c9e8fd470a940a114828eed387 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:07.779134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8558d-2500-4797-b61b-6e054fcb1b03 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:07.779134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8558d-2500-4797-b61b-6e054fcb1b03 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:07.779397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a8558d-2500-4797-b61b-6e054fcb1b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:07.793807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '268f0da98d8c4060a43cc4b70b239f85'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.805116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-67a8558d-2500-4797-b61b-6e054fcb1b03 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:07.807061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-67a8558d-2500-4797-b61b-6e054fcb1b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:07.906216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1132/2273] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:07.911389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22efe-83b2-401c-ae20-76ef29111a41 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:07.911682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22efe-83b2-401c-ae20-76ef29111a41 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:07.911974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee22efe-83b2-401c-ae20-76ef29111a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:07.915816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '268f0da98d8c4060a43cc4b70b239f85'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:07.923783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1142/2274] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:07 2022] DELETE /identity/v3/users/268f0da98d8c4060a43cc4b70b239f85 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:07.932354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-fee22efe-83b2-401c-ae20-76ef29111a41 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:07.933582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fee22efe-83b2-401c-ae20-76ef29111a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:08.023799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1133/2275] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:08.027701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.047030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f88f2d-08de-484b-bb03-567e5dcb08cb admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.055568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.056254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.056593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.057056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.057832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:08.058141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.060710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.081226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1143/2276] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:15:08.085832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22efe-83b2-401c-ae20-76ef29111a41 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.098737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee22efe-83b2-401c-ae20-76ef29111a41 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.103721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.104584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.104884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.105168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.105921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.107864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.210996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1134/2277] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:08 2022] POST /identity/v3/projects => generated 369 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:08.223877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.241662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f8e38-40af-428b-9eb6-6378670c71cf admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.312114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.313212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.313709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.314040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.314742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.317015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.443573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.445461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:08.458571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1144/2278] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:08 2022] POST /identity/v3/users => generated 285 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:08.462836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.475058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee974ec-5546-49a4-adcc-a490502a5e0d admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.478189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3f64-b7cb-4be8-949e-df64b6e3b9eb admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.486939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.487397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.488486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.488769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.489291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] PATH_INFO: `/v3/projects/0c4052c220d34f5fa2cde903771a5607` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.490319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c4052c220d34f5fa2cde903771a5607)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.491629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.492390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.495548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.495978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.501965 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.503748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.534026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1145/2279] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:08.535122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.554428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a59cf3-a424-4573-a19b-d3022290bc28 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.559662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.560920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.561243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.561580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.564952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.566791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.581655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1146/2280] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:08.586987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.604241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f166e9-7cb5-42f8-a9ab-0f963b72dfe5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.609704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9977530-d531-4e1d-aefc-4a84b4686c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.610819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9977530-d531-4e1d-aefc-4a84b4686c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.611715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9977530-d531-4e1d-aefc-4a84b4686c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.612005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9977530-d531-4e1d-aefc-4a84b4686c86 admin admin] PATH_INFO: `/v3/projects/5bf4c198860648cd874233ba27d1b971/users/577c7840904d4f2180f7e5bd0f530a9a/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.632334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9977530-d531-4e1d-aefc-4a84b4686c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf4c198860648cd874233ba27d1b971, user_id=577c7840904d4f2180f7e5bd0f530a9a, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.635316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9977530-d531-4e1d-aefc-4a84b4686c86 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.739221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1147/2281] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:08 2022] PUT /identity/v3/projects/5bf4c198860648cd874233ba27d1b971/users/577c7840904d4f2180f7e5bd0f530a9a/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:08.747215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.747509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.747789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.750241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c4052c220d34f5fa2cde903771a5607', 'request_id': 'req-7878f84b-68ef-4222-849e-1bb18830c1d0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:08.767739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:08.769060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:08.832991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1148/2282] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:08.845623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.845623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:08.858321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c4052c220d34f5fa2cde903771a5607', 'request_id': 'req-7878f84b-68ef-4222-849e-1bb18830c1d0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:08.867970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c4052c220d34f5fa2cde903771a5607', 'request_id': 'req-7878f84b-68ef-4222-849e-1bb18830c1d0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:08.884189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] The token cache is being invalidate because project 0c4052c220d34f5fa2cde903771a5607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:08.885626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:08.887007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7878f84b-68ef-4222-849e-1bb18830c1d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:08.888623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1135/2283] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:08 2022] DELETE /identity/v3/projects/0c4052c220d34f5fa2cde903771a5607 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:08.889515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:08.895528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874ba0d-b60b-4ac2-a829-c09f626fd1a2 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.895885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874ba0d-b60b-4ac2-a829-c09f626fd1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.896235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874ba0d-b60b-4ac2-a829-c09f626fd1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.912475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f221e8-60c7-4b3b-8674-737b8469b5b3 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.915917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9874ba0d-b60b-4ac2-a829-c09f626fd1a2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:08.917458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9874ba0d-b60b-4ac2-a829-c09f626fd1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:08.917920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.918884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.919170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.919511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.922222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:08.927751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1136/2284] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:08.936664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ae38-6327-4338-a9ca-1c4f00e69f0a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.940921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ae38-6327-4338-a9ca-1c4f00e69f0a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.940921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7ae38-6327-4338-a9ca-1c4f00e69f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.947117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.949525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:08.957052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e3b7ae38-6327-4338-a9ca-1c4f00e69f0a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:08.958432 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e3b7ae38-6327-4338-a9ca-1c4f00e69f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:08.964902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1149/2285] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:15:08.966717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1137/2286] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:08.971084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:08.980773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef65daa4-7533-439d-8977-a0a13e5ac83b service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:08.985937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:08.986627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:08.986877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:08.987149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:08.987924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:08.988211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:08.990149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.005308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1150/2287] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:15:09.009167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7ae38-6327-4338-a9ca-1c4f00e69f0a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:09.019149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7ae38-6327-4338-a9ca-1c4f00e69f0a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:09.023980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:09.024647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.024935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.025216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.025952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:09.028132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.163727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1138/2288] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:09 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:09.167988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:09.179892 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93caaee-3772-4599-bbf0-f3b8e9946c21 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:09.239887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:09.240589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.240857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.241135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.241890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:09.243896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.404477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1151/2289] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:09 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:09.406316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:09.417142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e02e8-421f-4920-af4a-a1efd22337ef admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:09.422978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:09.423742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.424037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.424325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.425073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:09.426782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.440516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1139/2290] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:09.444663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:09.455294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a891e8f-16ec-4d2e-83db-20c59615ea5d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:09.459898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:09.460494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.460777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.461051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.461851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:09.463464 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.474491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1152/2291] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:09.478775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:09.489775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af6eec-0bff-4910-b979-2b0f03f06b3e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:09.494376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f06214e-3b35-495c-84c0-02f4604704ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:09.495278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06214e-3b35-495c-84c0-02f4604704ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.495560 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06214e-3b35-495c-84c0-02f4604704ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.495834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f06214e-3b35-495c-84c0-02f4604704ed admin admin] PATH_INFO: `/v3/projects/8a0ad434ecc74c25b51326adda9d7a2f/users/c882f257ad6e46549a8d4442883e8161/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.511546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06214e-3b35-495c-84c0-02f4604704ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a0ad434ecc74c25b51326adda9d7a2f, user_id=c882f257ad6e46549a8d4442883e8161, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:09.514314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f06214e-3b35-495c-84c0-02f4604704ed admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.579838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:09.579838 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:09.585245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3e033-4de0-4687-82d9-7acbd4e95fdd admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:09.591476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:09.592400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.592795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.593709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] PATH_INFO: `/v3/users/e958d1c00e034c1281a3738c21e3e8ad` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.595693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=e958d1c00e034c1281a3738c21e3e8ad)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:09.598109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.653710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1140/2292] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:09 2022] PUT /identity/v3/projects/8a0ad434ecc74c25b51326adda9d7a2f/users/c882f257ad6e46549a8d4442883e8161/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:09.667285 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.667830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.668288 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.723937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] MFA Rules not processed for user `c882f257ad6e46549a8d4442883e8161`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:09.724130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:09.837150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1141/2293] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:09.858037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:09.860930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:09.930029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e958d1c00e034c1281a3738c21e3e8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:09.939034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e958d1c00e034c1281a3738c21e3e8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:09.939851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e958d1c00e034c1281a3738c21e3e8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:09.944057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec8a80-6138-44bd-b303-9b3a1c4b6755 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:09.948066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe08b560-f9f7-4e53-9e6f-c55d8669df0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:09.948755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08b560-f9f7-4e53-9e6f-c55d8669df0e service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:09.949140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08b560-f9f7-4e53-9e6f-c55d8669df0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:09.954495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe08b560-f9f7-4e53-9e6f-c55d8669df0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:09.958514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e958d1c00e034c1281a3738c21e3e8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:09.962392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08b560-f9f7-4e53-9e6f-c55d8669df0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:09.963891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe08b560-f9f7-4e53-9e6f-c55d8669df0e service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:09.978275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1142/2294] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:09.984636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:10.064245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e958d1c00e034c1281a3738c21e3e8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:10.070781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1153/2295] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:09 2022] DELETE /identity/v3/users/e958d1c00e034c1281a3738c21e3e8ad => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:10.075476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:10.075774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:10.076059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:10.094902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:10.096194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:10.162008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1143/2296] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:10.169109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:10.170201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:10.242929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0a2c3-3170-4680-8f78-31cedc0101ac admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:10.242929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e357e88-478d-4436-8c1b-63b9a03e0761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:10.243747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e357e88-478d-4436-8c1b-63b9a03e0761 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:10.243747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e357e88-478d-4436-8c1b-63b9a03e0761 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:10.243905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e357e88-478d-4436-8c1b-63b9a03e0761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:10.254528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e357e88-478d-4436-8c1b-63b9a03e0761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:10.256011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e357e88-478d-4436-8c1b-63b9a03e0761 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:10.271764 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1154/2297] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:10.850984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:10.852955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:10.877363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d59c5-cc09-44c1-a417-79c85492c56d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:10.920194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:10.921004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:10.922083 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:10.922222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] PATH_INFO: `/v3/projects/fc2ab1c7b6654b72959cd6d5faabe896` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:10.925148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc2ab1c7b6654b72959cd6d5faabe896)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:10.927236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:11.072598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc2ab1c7b6654b72959cd6d5faabe896', 'request_id': 'req-04492f6e-305f-400f-b01c-cff429593244'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:11.189669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc2ab1c7b6654b72959cd6d5faabe896', 'request_id': 'req-04492f6e-305f-400f-b01c-cff429593244'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:11.198557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc2ab1c7b6654b72959cd6d5faabe896', 'request_id': 'req-04492f6e-305f-400f-b01c-cff429593244'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:11.213683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] The token cache is being invalidate because project fc2ab1c7b6654b72959cd6d5faabe896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:11.214075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:11.214927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-04492f6e-305f-400f-b01c-cff429593244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:11.217292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1144/2298] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:10 2022] DELETE /identity/v3/projects/fc2ab1c7b6654b72959cd6d5faabe896 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:11.224388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a459e55-1a5f-41a1-91fc-b727e40b5f40 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.224388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a459e55-1a5f-41a1-91fc-b727e40b5f40 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.224388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a459e55-1a5f-41a1-91fc-b727e40b5f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:11.242698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9a459e55-1a5f-41a1-91fc-b727e40b5f40 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:11.243926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9a459e55-1a5f-41a1-91fc-b727e40b5f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:11.301769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:11.316752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1155/2299] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:11.319122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65898cd-5d0e-40a1-9399-4680d0fdffd0 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.319468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65898cd-5d0e-40a1-9399-4680d0fdffd0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.320152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65898cd-5d0e-40a1-9399-4680d0fdffd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:11.345946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b65898cd-5d0e-40a1-9399-4680d0fdffd0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:11.347342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b65898cd-5d0e-40a1-9399-4680d0fdffd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:11.356588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1145/2300] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:11.362121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a459e55-1a5f-41a1-91fc-b727e40b5f40 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:11.375003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a459e55-1a5f-41a1-91fc-b727e40b5f40 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:11.381122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:11.382064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.382369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.382651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:11.384093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:11.384409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:11.388818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:11.415395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1156/2301] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:15:11.428172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65898cd-5d0e-40a1-9399-4680d0fdffd0 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:11.438925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65898cd-5d0e-40a1-9399-4680d0fdffd0 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:11.445835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:11.447255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.451525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.451525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:11.451525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:11.453631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:11.632974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1146/2302] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:11 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:11.638109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:11.649434 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901035bd-020d-4e62-9ab0-2ec26080c091 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:11.750020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:11.751201 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.751617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.752003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:11.752852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:11.755040 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:11.873510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1157/2303] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:11 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:11.878943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:11.893772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142cfe1-d012-4e66-b666-e887ef6c9c19 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:11.899640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:11.900581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.901143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.901688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:11.902840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:11.904798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:11.923468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1147/2304] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:11.923960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:11.939165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb11d3-d438-415f-ba90-0ca30f8a6b51 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:11.944068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5176e6c-0929-42ab-a8bf-df33b5822424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:11.944972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176e6c-0929-42ab-a8bf-df33b5822424 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.945253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176e6c-0929-42ab-a8bf-df33b5822424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.945707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176e6c-0929-42ab-a8bf-df33b5822424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:11.946604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5176e6c-0929-42ab-a8bf-df33b5822424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:11.948456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5176e6c-0929-42ab-a8bf-df33b5822424 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:11.961083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1158/2305] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:15:11.968089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:11.980086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835dd565-84ba-4fea-a78e-37af757dbcd0 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:11.988897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:11.990014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:11.990403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:11.990778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] PATH_INFO: `/v3/projects/7e232959d574401bb4493058014f8f34/users/7ed5c56ec1a844fb994d8256ae7d7dc5/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:12.009017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e232959d574401bb4493058014f8f34, user_id=7ed5c56ec1a844fb994d8256ae7d7dc5, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:12.012141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:12.131921 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1148/2306] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:11 2022] PUT /identity/v3/projects/7e232959d574401bb4493058014f8f34/users/7ed5c56ec1a844fb994d8256ae7d7dc5/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:12.147697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44eaa-9351-478a-9907-dd239bd789e2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:12.148287 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44eaa-9351-478a-9907-dd239bd789e2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:12.148929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44eaa-9351-478a-9907-dd239bd789e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:12.181169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:12.207455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-95d44eaa-9351-478a-9907-dd239bd789e2 None None] MFA Rules not processed for user `7ed5c56ec1a844fb994d8256ae7d7dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:12.208683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-95d44eaa-9351-478a-9907-dd239bd789e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:12.295553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1159/2307] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:12.298727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:12.425310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3fba8-d3fc-4972-8925-e801242ff38a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:12.435197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:12.436188 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:12.436645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:12.436980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:12.440149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:12.443242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:12.646214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1149/2308] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:12 2022] POST /identity/v3/projects => generated 377 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:12.650204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d44eaa-9351-478a-9907-dd239bd789e2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:12.690667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d44eaa-9351-478a-9907-dd239bd789e2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:12.788296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:12.788296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:12.788296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:12.788296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:12.788673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:12.789929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:12.945286 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1160/2309] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:12 2022] POST /identity/v3/users => generated 289 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:12.949045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:12.960418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7681c502-88c6-474a-9194-1c0bf13e51de admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:12.966811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:12.967501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:12.967785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:12.968818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:12.969518 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:12.971284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:12.988451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1150/2310] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:12.992106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:13.003855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e59a1-82f2-4dc3-830b-287fe13b91c9 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:13.008992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8043c319-0579-4eed-941c-1a1b87a72165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:13.009716 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c319-0579-4eed-941c-1a1b87a72165 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:13.010006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c319-0579-4eed-941c-1a1b87a72165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:13.010285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8043c319-0579-4eed-941c-1a1b87a72165 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:13.011381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043c319-0579-4eed-941c-1a1b87a72165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:13.013009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8043c319-0579-4eed-941c-1a1b87a72165 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:13.025461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1161/2311] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:15:13.029710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:13.040637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9fe3d3-028a-42cd-8348-e2aeb9c653f8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:13.052171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:13.053300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:13.053731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:13.054116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] PATH_INFO: `/v3/projects/1323ced908d545e08f975c563a40f0a6/users/731e8319c0394315ba86f14696ec951b/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:13.073049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1323ced908d545e08f975c563a40f0a6, user_id=731e8319c0394315ba86f14696ec951b, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:13.075801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:13.087283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:13.145762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1151/2312] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:13 2022] PUT /identity/v3/projects/1323ced908d545e08f975c563a40f0a6/users/731e8319c0394315ba86f14696ec951b/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:13.158043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ff9dc-6435-4d39-8c67-b804da8cbe0f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:13.158608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ff9dc-6435-4d39-8c67-b804da8cbe0f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:13.158956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ff9dc-6435-4d39-8c67-b804da8cbe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:13.169555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:13.202443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6d0ff9dc-6435-4d39-8c67-b804da8cbe0f None None] MFA Rules not processed for user `731e8319c0394315ba86f14696ec951b`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:13.203825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6d0ff9dc-6435-4d39-8c67-b804da8cbe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:13.279742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1162/2313] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:13.285019 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:13.365630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9074df4-8721-4975-acd9-fc93a0ad4320 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:13.370737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:13.371675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:13.372142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:13.372451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:13.373156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:13.375669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:13.528854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1152/2314] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:13 2022] POST /identity/v3/projects => generated 373 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:15:13.533426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ff9dc-6435-4d39-8c67-b804da8cbe0f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:13.554215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ff9dc-6435-4d39-8c67-b804da8cbe0f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:13.672225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:13.673027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:13.673645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:13.674025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:13.676062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:13.678318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:13.797045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1163/2315] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:13 2022] POST /identity/v3/users => generated 293 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:13.814347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:13.814700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b2fca2-ba99-459f-bda3-f92ef1926d10 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:13.819645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:13.820327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:13.820613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:13.820897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:13.821883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:13.823585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:13.840000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1153/2316] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:13.845575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:13.859981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f1660-5a7a-41fd-95ba-665560ffc977 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:13.870121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:13.871192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:13.871565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:13.871821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] PATH_INFO: `/v3/projects/1ab5ab4d87fd4e71ac916e01d8bbcc67/users/363f9c894d9442bcb523416333b02467/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:13.924514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab5ab4d87fd4e71ac916e01d8bbcc67, user_id=363f9c894d9442bcb523416333b02467, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:13.927255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:14.018573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:14.019818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1164/2317] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:13 2022] PUT /identity/v3/projects/1ab5ab4d87fd4e71ac916e01d8bbcc67/users/363f9c894d9442bcb523416333b02467/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:14.030878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aef874-fa4c-4e2e-aedf-a6e002ab384a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:14.104687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9d02e-3c21-42c9-b3c9-66287f6e50c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:14.105519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9d02e-3c21-42c9-b3c9-66287f6e50c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:14.106005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9d02e-3c21-42c9-b3c9-66287f6e50c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:14.107368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9d02e-3c21-42c9-b3c9-66287f6e50c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:14.107368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9d02e-3c21-42c9-b3c9-66287f6e50c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:14.111556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9d02e-3c21-42c9-b3c9-66287f6e50c6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:14.129318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1154/2318] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:14.132121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:14.148632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4087b951-974c-4500-bb5d-9c8e41050aec admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:14.153734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:14.154646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:14.154930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:14.155204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] PATH_INFO: `/v3/projects/1ab5ab4d87fd4e71ac916e01d8bbcc67/users/363f9c894d9442bcb523416333b02467/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:14.158733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab5ab4d87fd4e71ac916e01d8bbcc67, user_id=363f9c894d9442bcb523416333b02467, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:14.162984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:14.185314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1165/2319] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:14 2022] PUT /identity/v3/projects/1ab5ab4d87fd4e71ac916e01d8bbcc67/users/363f9c894d9442bcb523416333b02467/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:14.193392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723f10-4606-4471-8e0a-0b9d7f3bc7de None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:14.194278 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723f10-4606-4471-8e0a-0b9d7f3bc7de None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:14.194573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87723f10-4606-4471-8e0a-0b9d7f3bc7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:14.243075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-87723f10-4606-4471-8e0a-0b9d7f3bc7de None None] MFA Rules not processed for user `363f9c894d9442bcb523416333b02467`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:14.244946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-87723f10-4606-4471-8e0a-0b9d7f3bc7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:14.328120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1155/2320] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:14 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:14.340654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:14.342383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:14.416159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b59485-6c0d-4e9f-8321-f0522efff270 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:14.420326 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:14.422098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:14.422741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:14.423411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:14.505089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:14.507600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:14.523115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1166/2321] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:14 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:14.934681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:16.399876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd36aa-649a-4362-89ca-93dda872b4a2 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:16.400502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd36aa-649a-4362-89ca-93dda872b4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:16.400687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd36aa-649a-4362-89ca-93dda872b4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:16.448093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:16.508917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c8cd36aa-649a-4362-89ca-93dda872b4a2 None None] MFA Rules not processed for user `577c7840904d4f2180f7e5bd0f530a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:16.512546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c8cd36aa-649a-4362-89ca-93dda872b4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:16.693792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1156/2322] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:16.698191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:16.700003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:16.885710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50843e82-92a5-43e1-bf7e-57b4befea50c service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:16.892391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:16.893153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:16.893555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:16.893903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:16.894687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:16.896789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:17.063777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1167/2323] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:16 2022] POST /identity/v3/projects => generated 369 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:17.065925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd36aa-649a-4362-89ca-93dda872b4a2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:17.077537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd36aa-649a-4362-89ca-93dda872b4a2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:17.198903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:17.199590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:17.199907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:17.202842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:17.204681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:17.206667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:17.373391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1157/2324] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:17 2022] POST /identity/v3/users => generated 291 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:15:17.379582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:17.395724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f304f-c12b-4870-bb6e-5b06b066b030 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:17.403768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:17.405199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:17.406335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:17.406673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:17.407697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:17.410783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:17.425997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1168/2325] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:17.427490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:17.442693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eabe99-b0c7-45ba-9a69-92671ed07619 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:17.449487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:17.450629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:17.451202 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:17.451589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] PATH_INFO: `/v3/projects/2e343093ea7146318468d8c7e39f3703/users/77fc5b985da047a7aec85bc3aa4f1957/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:17.476964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e343093ea7146318468d8c7e39f3703, user_id=77fc5b985da047a7aec85bc3aa4f1957, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:17.481756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:17.591137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1158/2326] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:17 2022] PUT /identity/v3/projects/2e343093ea7146318468d8c7e39f3703/users/77fc5b985da047a7aec85bc3aa4f1957/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:17.595943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:17.610407 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc20054-3a19-4a34-ac40-b3f9fca1d6b6 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:17.689107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:17.689810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:17.690093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:17.690370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:17.691140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:17.692753 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:17.703916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1169/2327] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:17.708075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:17.729444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63837d-d9c9-467b-818c-64b8556bd0f9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:17.738006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:17.739258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:17.739690 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:17.740111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] PATH_INFO: `/v3/projects/2e343093ea7146318468d8c7e39f3703/users/77fc5b985da047a7aec85bc3aa4f1957/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:17.742380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e343093ea7146318468d8c7e39f3703, user_id=77fc5b985da047a7aec85bc3aa4f1957, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:17.745166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:17.758803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1159/2328] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:17 2022] PUT /identity/v3/projects/2e343093ea7146318468d8c7e39f3703/users/77fc5b985da047a7aec85bc3aa4f1957/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:17.975796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:17.987647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:18.058838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:18.130654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cda66-c612-4197-b1f5-cd007ea9389f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:18.210015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7406460-0cb6-45d0-8143-c0d4015d34b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:18.211205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7406460-0cb6-45d0-8143-c0d4015d34b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:18.211205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7406460-0cb6-45d0-8143-c0d4015d34b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:18.211484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7406460-0cb6-45d0-8143-c0d4015d34b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:18.277995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7406460-0cb6-45d0-8143-c0d4015d34b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:18.279566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7406460-0cb6-45d0-8143-c0d4015d34b9 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:18.289995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1170/2329] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:20.784805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:20.908600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:21.586127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:21.608465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77832965-d63c-480d-a391-ffa6515c65d9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:21.612989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:21.615825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:21.615825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:21.615825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:21.719354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:21.720720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:21.731331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1160/2330] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:22.979513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:24.127479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:24.143378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:25.049211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:25.049524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:25.049801 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:25.076672 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] MFA Rules not processed for user `77fc5b985da047a7aec85bc3aa4f1957`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:25.078056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:25.144285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1171/2331] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:25 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:25.158444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:25.159870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:25.239776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4d646-393d-4d14-bcb6-f074b53d12d5 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:25.245985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d1d6ed-fe4f-4162-9716-2ef3238219d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:25.246876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d6ed-fe4f-4162-9716-2ef3238219d3 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:25.247165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d6ed-fe4f-4162-9716-2ef3238219d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:25.247443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d1d6ed-fe4f-4162-9716-2ef3238219d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:25.292355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:25.308581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1d6ed-fe4f-4162-9716-2ef3238219d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:25.310223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d1d6ed-fe4f-4162-9716-2ef3238219d3 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:25.320065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1161/2332] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:26.080801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:29.075325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:29.198194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:29.200037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:29.312447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578fa40-0bc1-4a74-a5bf-690a656906df None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:29.315954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:29.317732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:29.318325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:29.318863 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] PATH_INFO: `/v3/users/c882f257ad6e46549a8d4442883e8161` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:29.320235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c882f257ad6e46549a8d4442883e8161)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:29.322344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:29.692185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17ab8-5a50-43ca-b7fe-92a4bb745c33 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:29.692738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17ab8-5a50-43ca-b7fe-92a4bb745c33 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:29.693231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c17ab8-5a50-43ca-b7fe-92a4bb745c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:29.706943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c882f257ad6e46549a8d4442883e8161'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:29.716840 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c882f257ad6e46549a8d4442883e8161'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:29.718452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-62c17ab8-5a50-43ca-b7fe-92a4bb745c33 None None] MFA Rules not processed for user `577c7840904d4f2180f7e5bd0f530a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:29.719801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-62c17ab8-5a50-43ca-b7fe-92a4bb745c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:29.725607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c882f257ad6e46549a8d4442883e8161'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:29.739601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1162/2333] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:29.743374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c882f257ad6e46549a8d4442883e8161'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:29.851806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c882f257ad6e46549a8d4442883e8161'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:29.860215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1172/2334] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:29 2022] DELETE /identity/v3/users/c882f257ad6e46549a8d4442883e8161 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:29.870045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:29.870335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:29.870653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:29.897814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:29.899054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:29.986082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1163/2335] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:29.994780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:29.996395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:30.106653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e88c8e-0267-446d-adf3-1aece4150dae admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:30.112328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:30.113241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:30.113769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:30.117983 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:30.129346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:30.130809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:30.154663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1173/2336] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:30.441200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:30.920244 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:30.922824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:30.949561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ba545-c530-4bc3-8fd2-b5cbb836905f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:30.975738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:30.975738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:30.975738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:30.975738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] PATH_INFO: `/v3/projects/8a0ad434ecc74c25b51326adda9d7a2f` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:30.976888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a0ad434ecc74c25b51326adda9d7a2f)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:30.978949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:31.157324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:31.160213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:31.176718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64b4e9-ac3e-4e23-928f-4e0752a36940 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:31.193900 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a0ad434ecc74c25b51326adda9d7a2f', 'request_id': 'req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.261752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:31.262940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:31.263322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:31.263743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] PATH_INFO: `/v3/users/7ed5c56ec1a844fb994d8256ae7d7dc5` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:31.265742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed5c56ec1a844fb994d8256ae7d7dc5)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:31.269180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:31.286219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a0ad434ecc74c25b51326adda9d7a2f', 'request_id': 'req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.293587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a0ad434ecc74c25b51326adda9d7a2f', 'request_id': 'req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.310847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] The token cache is being invalidate because project 8a0ad434ecc74c25b51326adda9d7a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:31.311264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.312141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c85c62a2-3753-4ae4-b122-eab88dd5ffa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.314724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1164/2337] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:30 2022] DELETE /identity/v3/projects/8a0ad434ecc74c25b51326adda9d7a2f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:31.319728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e951d0-1e5a-488c-9a15-99aa003ae3a3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:31.320117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e951d0-1e5a-488c-9a15-99aa003ae3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:31.320497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e951d0-1e5a-488c-9a15-99aa003ae3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:31.379211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-10e951d0-1e5a-488c-9a15-99aa003ae3a3 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:31.380594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-10e951d0-1e5a-488c-9a15-99aa003ae3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:31.390331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1165/2338] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:31.398341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348598f-ac5a-4df6-9224-8f526789381a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:31.399193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348598f-ac5a-4df6-9224-8f526789381a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:31.399675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348598f-ac5a-4df6-9224-8f526789381a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:31.430495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f348598f-ac5a-4df6-9224-8f526789381a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:31.431830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f348598f-ac5a-4df6-9224-8f526789381a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:31.444337 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1166/2339] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:31.459792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348598f-ac5a-4df6-9224-8f526789381a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:31.470625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f348598f-ac5a-4df6-9224-8f526789381a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:31.475703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:31.476475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:31.476760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:31.477323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:31.479147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:31.479635 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:31.481719 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:31.499185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1167/2340] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:15:31.503999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:31.519162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9458414-b5da-4746-8a85-f522a77dbfc7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:31.524357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:31.525402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:31.525632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:31.525908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:31.526613 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:31.528536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:31.643631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed5c56ec1a844fb994d8256ae7d7dc5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.656018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed5c56ec1a844fb994d8256ae7d7dc5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.657146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed5c56ec1a844fb994d8256ae7d7dc5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.670374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed5c56ec1a844fb994d8256ae7d7dc5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.712811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1168/2341] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:31 2022] POST /identity/v3/projects => generated 359 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:31.718174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:31.801514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d9d4d3-3dcc-45ff-81d6-84df606e27e6 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:31.812391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:31.814073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:31.814439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:31.814907 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:31.816705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:31.820138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:31.888670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-f537cbb1-6d88-4328-bd89-c986d86369e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed5c56ec1a844fb994d8256ae7d7dc5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:31.898820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1174/2342] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:31 2022] DELETE /identity/v3/users/7ed5c56ec1a844fb994d8256ae7d7dc5 => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:31.903244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:31.903612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:31.903901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:31.932297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:31.933964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:31.954322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1169/2343] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:31 2022] POST /identity/v3/users => generated 280 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:31.957121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:31.960206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:31.972808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c099020-f043-45ea-b2c5-8dc13d1435cb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:32.016848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1175/2344] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:32.046679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:32.049043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:32.100151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:32.101345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.102436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.103311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.104537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:32.107265 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:32.132173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1170/2345] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:32.169653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd8536f-a3bf-4af7-b5d3-52e3e89582ad None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:32.179816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:32.179816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.179816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.179816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.180227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:32.201458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:32.203060 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:32.212123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2207fe-400c-4ec5-84d6-f3dbb08a9710 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:32.221125 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490ee5c7-b88f-44ad-91e3-bd7fe8c4df3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:32.222257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee5c7-b88f-44ad-91e3-bd7fe8c4df3a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.222799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee5c7-b88f-44ad-91e3-bd7fe8c4df3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.223302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490ee5c7-b88f-44ad-91e3-bd7fe8c4df3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.224294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ee5c7-b88f-44ad-91e3-bd7fe8c4df3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:32.227176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490ee5c7-b88f-44ad-91e3-bd7fe8c4df3a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:32.240278 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1176/2346] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:32.250328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1171/2347] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:32.254580 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:32.271522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617be6e-068d-44c9-878d-45727f0aa9ec service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:32.278829 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:32.289404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.289404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.289404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] PATH_INFO: `/v3/projects/11f1a5cee57b4f8680da2844b7823fd4/users/0844d41fa6a24c3d9a48bc77c9b7fbff/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.301566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f1a5cee57b4f8680da2844b7823fd4, user_id=0844d41fa6a24c3d9a48bc77c9b7fbff, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:32.304416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:32.475445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1177/2348] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:32 2022] PUT /identity/v3/projects/11f1a5cee57b4f8680da2844b7823fd4/users/0844d41fa6a24c3d9a48bc77c9b7fbff/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:32.486767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbc79d-e425-44fa-b337-7ef9cafc37f8 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.487093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbc79d-e425-44fa-b337-7ef9cafc37f8 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.487379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbc79d-e425-44fa-b337-7ef9cafc37f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.515293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-38cbc79d-e425-44fa-b337-7ef9cafc37f8 None None] MFA Rules not processed for user `0844d41fa6a24c3d9a48bc77c9b7fbff`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:32.516538 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-38cbc79d-e425-44fa-b337-7ef9cafc37f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:32.588671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1172/2349] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:32 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:32.621258 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:32.623320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:32.688585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bf5c5-c48c-47f5-b2a3-133a947bcf49 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:32.692351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:32.693010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.693316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.693593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.703068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:32.704545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:32.714171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1178/2350] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:32 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:15:32.732050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbc79d-e425-44fa-b337-7ef9cafc37f8 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:32.798380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbc79d-e425-44fa-b337-7ef9cafc37f8 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:32.803041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:32.803959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.804261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.804700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] PATH_INFO: `/v3/users/0844d41fa6a24c3d9a48bc77c9b7fbff` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.805893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0844d41fa6a24c3d9a48bc77c9b7fbff)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:32.807847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:32.866096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:32.868366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:32.887942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f56a5d-15cf-4e6b-a3d6-10eb1d26dcc4 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:32.890371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:32.898472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:32.899374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:32.899611 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:32.899889 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] PATH_INFO: `/v3/projects/7e232959d574401bb4493058014f8f34` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:32.901210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e232959d574401bb4493058014f8f34)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:32.903299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:33.044269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0844d41fa6a24c3d9a48bc77c9b7fbff'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.045235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0844d41fa6a24c3d9a48bc77c9b7fbff'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.050405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e232959d574401bb4493058014f8f34', 'request_id': 'req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.053298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0844d41fa6a24c3d9a48bc77c9b7fbff'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.053999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0844d41fa6a24c3d9a48bc77c9b7fbff'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.068116 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-c7995b9a-9870-483f-a61a-631f7e9f961a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0844d41fa6a24c3d9a48bc77c9b7fbff'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.183528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e232959d574401bb4493058014f8f34', 'request_id': 'req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.191422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e232959d574401bb4493058014f8f34', 'request_id': 'req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.203532 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] The token cache is being invalidate because project 7e232959d574401bb4493058014f8f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:33.203933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.204668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.206669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1179/2351] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:32 2022] DELETE /identity/v3/projects/7e232959d574401bb4493058014f8f34 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:33.210281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:33.212148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:33.289664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1173/2352] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:32 2022] DELETE /identity/v3/users/0844d41fa6a24c3d9a48bc77c9b7fbff => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:33.291992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:33.292290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:33.292567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:33.301853 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49a5d4-72b3-4456-9bc0-b35fecf3f7d5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:33.336732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:33.338022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:33.414476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:33.417178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:33.417581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:33.417994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] PATH_INFO: `/v3/users/731e8319c0394315ba86f14696ec951b` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:33.419766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=731e8319c0394315ba86f14696ec951b)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:33.423964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:33.445786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1174/2353] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:33.471063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:33.472511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:33.485191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce0fcd-7655-4f20-b27d-a3a45889a22a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:33.491173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:33.492056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:33.492345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:33.493216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] PATH_INFO: `/v3/users/8d4b191d60834b8db48549a9f13c7d97` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:33.494524 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4b191d60834b8db48549a9f13c7d97)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:33.496466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:33.683202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '731e8319c0394315ba86f14696ec951b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.693200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '731e8319c0394315ba86f14696ec951b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.694116 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '731e8319c0394315ba86f14696ec951b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.706866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '731e8319c0394315ba86f14696ec951b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.752270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:33.766234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4b191d60834b8db48549a9f13c7d97'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.766993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4b191d60834b8db48549a9f13c7d97'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.774664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4b191d60834b8db48549a9f13c7d97'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.775404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4b191d60834b8db48549a9f13c7d97'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.778185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '731e8319c0394315ba86f14696ec951b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.786487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1180/2354] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:33 2022] DELETE /identity/v3/users/731e8319c0394315ba86f14696ec951b => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:33.787424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:33.788883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:33.791095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f94b69b9-7e6f-49f5-8e4b-2ffaf309953e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4b191d60834b8db48549a9f13c7d97'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:33.852518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05befd1-1334-411e-9f9c-d812a8e8ada3 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:33.856703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:33.857426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:33.857719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:33.858876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:33.861104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:33.871623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1175/2355] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:33 2022] DELETE /identity/v3/users/8d4b191d60834b8db48549a9f13c7d97 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:33.876247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:33.876725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:33.877369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:33.899453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:33.900803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:33.939971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:33.941544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:33.952791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1181/2356] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:33.974674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1176/2357] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:33.986710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:33.995516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa73fa-03be-4b92-8678-08bf7635f039 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:34.056893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:34.057734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:34.058018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:34.058296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:34.076229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:34.078120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:34.088513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1182/2358] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:34.640102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:34.641722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:34.653546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ea8ea-82e3-45d2-90ff-8ef2b7274739 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:34.658852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:34.659713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:34.660005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:34.660275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] PATH_INFO: `/v3/projects/1323ced908d545e08f975c563a40f0a6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:34.662228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1323ced908d545e08f975c563a40f0a6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:34.663795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:34.847578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:34.854478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:34.892429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f803c7-d0fe-4941-94b9-ea3ea6e9962e service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:34.928583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1323ced908d545e08f975c563a40f0a6', 'request_id': 'req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:34.977310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:34.981653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:34.981653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:34.981653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] PATH_INFO: `/v3/projects/11f1a5cee57b4f8680da2844b7823fd4` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:34.981653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f1a5cee57b4f8680da2844b7823fd4)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:34.983013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:35.006515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1323ced908d545e08f975c563a40f0a6', 'request_id': 'req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.020998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1323ced908d545e08f975c563a40f0a6', 'request_id': 'req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.032798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] The token cache is being invalidate because project 1323ced908d545e08f975c563a40f0a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:35.033169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.033848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.035816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1177/2359] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:34 2022] DELETE /identity/v3/projects/1323ced908d545e08f975c563a40f0a6 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:35.039521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:35.043479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.054714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bbcf6-14ed-4a7c-a6b8-b53120dcef38 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:35.059618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:35.060468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.060745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.061627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] PATH_INFO: `/v3/users/363f9c894d9442bcb523416333b02467` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.062991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] RBAC: Authorizing `identity:delete_user(user_id=363f9c894d9442bcb523416333b02467)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:35.064895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:35.152238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f1a5cee57b4f8680da2844b7823fd4', 'request_id': 'req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.211585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f1a5cee57b4f8680da2844b7823fd4', 'request_id': 'req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.218598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f1a5cee57b4f8680da2844b7823fd4', 'request_id': 'req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.229972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] The token cache is being invalidate because project 11f1a5cee57b4f8680da2844b7823fd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:35.230229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.230876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.232600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1183/2360] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:34 2022] DELETE /identity/v3/projects/11f1a5cee57b4f8680da2844b7823fd4 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:35.234253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:35.236105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.303594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b7e97-58b4-4703-a1b8-083a0cd7bd7d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:35.308562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:35.309670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.309960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.318904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] PATH_INFO: `/v3/projects/d21f3fc90f984d44bd75cc3fd7ebbc09` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.318904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d21f3fc90f984d44bd75cc3fd7ebbc09)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:35.318904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:35.364740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363f9c894d9442bcb523416333b02467'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.365477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363f9c894d9442bcb523416333b02467'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.373440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363f9c894d9442bcb523416333b02467'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.374822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363f9c894d9442bcb523416333b02467'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.391257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e233635-d932-4ea4-9bc1-5b509e20919c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363f9c894d9442bcb523416333b02467'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.464447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd21f3fc90f984d44bd75cc3fd7ebbc09', 'request_id': 'req-d33e2ba1-078f-434e-986c-5e8002290554'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.515893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1178/2361] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:35 2022] DELETE /identity/v3/users/363f9c894d9442bcb523416333b02467 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:35.517200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46c94a-54be-4213-9aad-b33092fa4281 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.517576 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46c94a-54be-4213-9aad-b33092fa4281 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.517806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46c94a-54be-4213-9aad-b33092fa4281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.550472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4b46c94a-54be-4213-9aad-b33092fa4281 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:35.552793 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4b46c94a-54be-4213-9aad-b33092fa4281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.626519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1179/2362] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:35.634973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98613eb-8242-4baf-9f1d-8208fb00e2a5 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.635262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98613eb-8242-4baf-9f1d-8208fb00e2a5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.635541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98613eb-8242-4baf-9f1d-8208fb00e2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.653901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c98613eb-8242-4baf-9f1d-8208fb00e2a5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:35.655302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c98613eb-8242-4baf-9f1d-8208fb00e2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.663140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1180/2363] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:35.667570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98613eb-8242-4baf-9f1d-8208fb00e2a5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:35.677147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98613eb-8242-4baf-9f1d-8208fb00e2a5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:35.681908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:35.682566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.682853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.683134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.683862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:35.684175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:35.685823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:35.697366 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd21f3fc90f984d44bd75cc3fd7ebbc09', 'request_id': 'req-d33e2ba1-078f-434e-986c-5e8002290554'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.698238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1181/2364] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:15:35.703060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:35.705446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd21f3fc90f984d44bd75cc3fd7ebbc09', 'request_id': 'req-d33e2ba1-078f-434e-986c-5e8002290554'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.712825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4825a-8cc4-4978-9851-e50a425aef78 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:35.715857 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] The token cache is being invalidate because project d21f3fc90f984d44bd75cc3fd7ebbc09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:35.716162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.716792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d33e2ba1-078f-434e-986c-5e8002290554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:35.718546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:35.718710 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1184/2365] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:35 2022] DELETE /identity/v3/projects/d21f3fc90f984d44bd75cc3fd7ebbc09 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:35.719454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.719760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.720118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.720822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:35.723218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07825226-f481-45d0-99f4-193eac22775d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.723362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:35.723514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07825226-f481-45d0-99f4-193eac22775d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.723985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07825226-f481-45d0-99f4-193eac22775d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.757561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-07825226-f481-45d0-99f4-193eac22775d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:35.759001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-07825226-f481-45d0-99f4-193eac22775d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.772043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1185/2366] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:35.781457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b8d50-c3cc-4fff-99d5-001b888da25f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.781457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b8d50-c3cc-4fff-99d5-001b888da25f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.781722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b8d50-c3cc-4fff-99d5-001b888da25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.806240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-355b8d50-c3cc-4fff-99d5-001b888da25f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:35.807446 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-355b8d50-c3cc-4fff-99d5-001b888da25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.816410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1186/2367] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:35.821741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ebb1a-7c45-4557-8793-a5e8c9c1aa10 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:35.821741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ebb1a-7c45-4557-8793-a5e8c9c1aa10 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:35.821741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ebb1a-7c45-4557-8793-a5e8c9c1aa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:35.836609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1f7ebb1a-7c45-4557-8793-a5e8c9c1aa10 None None] MFA Rules not processed for user `6e8f1cb9760f41c6b7c1eb379daf1b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:35.837864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1f7ebb1a-7c45-4557-8793-a5e8c9c1aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.909848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1187/2368] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:35.914603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1182/2369] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:35 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:15:35.914727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ebb1a-7c45-4557-8793-a5e8c9c1aa10 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:35.918396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:35.924445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:35.925553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ebb1a-7c45-4557-8793-a5e8c9c1aa10 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.030643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:36.035427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.036549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.036549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.036751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.038736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:36.038736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.042066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.077239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1188/2370] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:15:36.078161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:36.080375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.080782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff904a5-4444-4187-953b-6fa2267a0d0c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.087503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.088221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.088704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.089164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.090519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.096507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.099510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2126372-01af-4ebd-afd0-d9fd0377e4d2 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.107553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.109750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.110037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.110317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] PATH_INFO: `/v3/users/8481753694d2460b85a1089704f87640` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.112158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8481753694d2460b85a1089704f87640)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.115718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.187349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1183/2371] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:35 2022] POST /identity/v3/users => generated 289 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:36.188507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:36.190730 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.216476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85de0d-0986-4fb7-91a8-7e6882a7debf admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.222845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.223919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.224208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.224484 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] PATH_INFO: `/v3/projects/1ab5ab4d87fd4e71ac916e01d8bbcc67` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.225892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab5ab4d87fd4e71ac916e01d8bbcc67)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.227903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.485654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab5ab4d87fd4e71ac916e01d8bbcc67', 'request_id': 'req-bead3e1c-1fdf-43b0-84a7-4543e98270c5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.502027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8481753694d2460b85a1089704f87640'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.511232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8481753694d2460b85a1089704f87640'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.511232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8481753694d2460b85a1089704f87640'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.523655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8481753694d2460b85a1089704f87640'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.560292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab5ab4d87fd4e71ac916e01d8bbcc67', 'request_id': 'req-bead3e1c-1fdf-43b0-84a7-4543e98270c5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.572060 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab5ab4d87fd4e71ac916e01d8bbcc67', 'request_id': 'req-bead3e1c-1fdf-43b0-84a7-4543e98270c5'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.584178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] The token cache is being invalidate because project 1ab5ab4d87fd4e71ac916e01d8bbcc67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:36.584464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.585112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.586687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1184/2372] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:36 2022] DELETE /identity/v3/projects/1ab5ab4d87fd4e71ac916e01d8bbcc67 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:36.587953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:36.589441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.652258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead3e1c-1fdf-43b0-84a7-4543e98270c5 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.657170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddfe1f-ae83-434d-9357-669a663f2cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.657853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddfe1f-ae83-434d-9357-669a663f2cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.658135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddfe1f-ae83-434d-9357-669a663f2cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.658410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddfe1f-ae83-434d-9357-669a663f2cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.659096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddfe1f-ae83-434d-9357-669a663f2cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.661068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddfe1f-ae83-434d-9357-669a663f2cf7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.685539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8481753694d2460b85a1089704f87640'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:36.694975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1189/2373] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:36 2022] DELETE /identity/v3/users/8481753694d2460b85a1089704f87640 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:36.695867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:36.697257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.786502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1185/2374] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:36 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:36.788489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.788925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.790327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.792581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0ff10-bdd5-4643-8414-062d269f856d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.811525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:36.812969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.846101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:36.866083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b37c3-5338-4606-973e-7ffe47951c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.866914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b37c3-5338-4606-973e-7ffe47951c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.867203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b37c3-5338-4606-973e-7ffe47951c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.867483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b37c3-5338-4606-973e-7ffe47951c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.868212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b37c3-5338-4606-973e-7ffe47951c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.869925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b37c3-5338-4606-973e-7ffe47951c90 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.881539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1190/2375] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:36.887026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b370c9-65ba-468a-96c4-320ea91285af None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.887026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b370c9-65ba-468a-96c4-320ea91285af None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.887026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b370c9-65ba-468a-96c4-320ea91285af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.898607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1186/2376] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:36.898607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:36.899847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.906257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-00b370c9-65ba-468a-96c4-320ea91285af None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:36.907833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-00b370c9-65ba-468a-96c4-320ea91285af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.914895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a82871-01be-4fd0-a4f6-1822285ceb2a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.920098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.920685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.920978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.921308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.922328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.922648 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1191/2377] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:36.923589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b370c9-65ba-468a-96c4-320ea91285af None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:36.927492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.937735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b370c9-65ba-468a-96c4-320ea91285af None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:36.942667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb5a17-c8e6-43ab-b854-fcb19c5fe0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:36.943422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb5a17-c8e6-43ab-b854-fcb19c5fe0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.943701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb5a17-c8e6-43ab-b854-fcb19c5fe0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.943974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb5a17-c8e6-43ab-b854-fcb19c5fe0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.944703 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb5a17-c8e6-43ab-b854-fcb19c5fe0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:36.946318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb5a17-c8e6-43ab-b854-fcb19c5fe0b0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:36.957568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1192/2378] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:36.958985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:36.959266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:36.959538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:36.978361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:36.979739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.991098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1193/2379] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:36.991993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:36.993279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:36.997995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1187/2380] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:36 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:36.997995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.009955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4cd691-888b-4790-8f67-224c74674d45 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.015494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.016461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.016809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.017151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] PATH_INFO: `/v3/projects/cd01cba058564916a3422acc5bbec080/users/82803409c2344e379035e0fb3ee6b023/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.033391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd01cba058564916a3422acc5bbec080, user_id=82803409c2344e379035e0fb3ee6b023, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.036151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.058728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c0f48-ea88-4c40-b6e9-dbb6d76f5e29 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.063557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.064579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.064998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.065455 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.082386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.083858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.095371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1194/2381] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:37.096155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.109480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8e34a-79c3-4455-891c-4ff49d1ca099 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.110971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1188/2382] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:36 2022] PUT /identity/v3/projects/cd01cba058564916a3422acc5bbec080/users/82803409c2344e379035e0fb3ee6b023/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:37.112076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.188016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b12567-d791-478b-9afd-055195905fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.188799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b12567-d791-478b-9afd-055195905fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.189085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b12567-d791-478b-9afd-055195905fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.189503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b12567-d791-478b-9afd-055195905fb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.190226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b12567-d791-478b-9afd-055195905fb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:37.190538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b12567-d791-478b-9afd-055195905fb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.192436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b12567-d791-478b-9afd-055195905fb4 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.209001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee4922-0643-4389-8f1e-b655d6cb1432 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.213482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1195/2383] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:15:37.214180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.214969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.215440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.215558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099ea12-3c34-4a57-8275-1fe56a087ee6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.216158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099ea12-3c34-4a57-8275-1fe56a087ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.216654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099ea12-3c34-4a57-8275-1fe56a087ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.241982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.243429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.247851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.261958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f099ea12-3c34-4a57-8275-1fe56a087ee6 None None] MFA Rules not processed for user `82803409c2344e379035e0fb3ee6b023`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:37.267005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f099ea12-3c34-4a57-8275-1fe56a087ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:37.278136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1189/2384] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:37.279127 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.296135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7f8c0-28d2-4162-a653-011315ba440c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.304145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.304826 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.305117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.305439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.306235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.308658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.350338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1196/2385] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:37.351216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099ea12-3c34-4a57-8275-1fe56a087ee6 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.371248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099ea12-3c34-4a57-8275-1fe56a087ee6 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.376383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.377075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.377974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.377974 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.378589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.380211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.391483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1197/2386] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:37.392493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.395435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:37.454716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1190/2387] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:37 2022] POST /identity/v3/projects => generated 373 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:37.458529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.502285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405747d-8158-426b-b43c-9d4674ac527b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.556003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd63b87-3243-4657-83bc-7be184cce71b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.563953 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.571362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.571362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.571362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] PATH_INFO: `/v3/projects/ef6cbf12b58d488b88c1ac4dd0d45712/users/d8ca99e72c0f4e48a38a84e4858dbdde/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.587228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6cbf12b58d488b88c1ac4dd0d45712, user_id=d8ca99e72c0f4e48a38a84e4858dbdde, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.590065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.608846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.609646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.609997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.610438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.676657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.678124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.686988 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1198/2388] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:37.687754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.697577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c259ea-f8c5-45dc-a0fd-cf70bdfb566b service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.703778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b5aee3-e7c0-400c-ab5c-f507866d5300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.704554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5aee3-e7c0-400c-ab5c-f507866d5300 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.706948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5aee3-e7c0-400c-ab5c-f507866d5300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.706948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b5aee3-e7c0-400c-ab5c-f507866d5300 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.706948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5aee3-e7c0-400c-ab5c-f507866d5300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.708087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b5aee3-e7c0-400c-ab5c-f507866d5300 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.736638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1191/2389] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:37 2022] PUT /identity/v3/projects/ef6cbf12b58d488b88c1ac4dd0d45712/users/d8ca99e72c0f4e48a38a84e4858dbdde/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:37.738122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.746676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:37.757810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:37.827523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1199/2390] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:37 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:37.830001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cc77b-0507-4755-8d68-951c3c908ca2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.830296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cc77b-0507-4755-8d68-951c3c908ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.830727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7cc77b-0507-4755-8d68-951c3c908ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.843329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b1817-7443-44fc-841f-50c4df0ebadb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.849931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.851055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.851498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.851931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] PATH_INFO: `/v3/projects/be4535132aa5456c99ac0b602e8a7d17` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.853522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be4535132aa5456c99ac0b602e8a7d17)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.856089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.866634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5d7cc77b-0507-4755-8d68-951c3c908ca2 None None] MFA Rules not processed for user `d8ca99e72c0f4e48a38a84e4858dbdde`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:37.867960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5d7cc77b-0507-4755-8d68-951c3c908ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:37.959404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1200/2391] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:37.960372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cc77b-0507-4755-8d68-951c3c908ca2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.971466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7cc77b-0507-4755-8d68-951c3c908ca2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:37.976094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:37.976771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:37.977165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:37.977481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:37.978217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:37.979821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:37.989996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1201/2392] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:37.990781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:37.992400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:38.050653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a80e718-1053-4368-9c28-cebef27cbf1b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.056526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.057109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.057426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.057707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.073183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.075124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.088733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1202/2393] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:38.089614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.105131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c608e67-bf1d-4b0a-8a82-e914d41bcdde service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.119112 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be4535132aa5456c99ac0b602e8a7d17', 'request_id': 'req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:38.169517 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.170739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.173647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.173993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.174774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.176702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.189247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1203/2394] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:38.194360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.206223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be4535132aa5456c99ac0b602e8a7d17', 'request_id': 'req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:38.214132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be4535132aa5456c99ac0b602e8a7d17', 'request_id': 'req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:38.218719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87fa1bf-1307-4e6c-ba19-2fa5704ce15f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.228405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] The token cache is being invalidate because project be4535132aa5456c99ac0b602e8a7d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:38.228767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:38.229555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-7a42cd38-bd83-4ccf-b3e3-6846dd0e96b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:38.231266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1192/2395] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:37 2022] DELETE /identity/v3/projects/be4535132aa5456c99ac0b602e8a7d17 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:38.234728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a63cce1-9265-4005-add1-c8c766ddf1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.235793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cce1-9265-4005-add1-c8c766ddf1e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.236083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cce1-9265-4005-add1-c8c766ddf1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.236993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a63cce1-9265-4005-add1-c8c766ddf1e7 admin admin] PATH_INFO: `/v3/projects/8a41869fa0bf49ea8f684f7585d96578/users/339d9b743e15476480900fdeb90f694f/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.237180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c67f4-d009-4852-89d2-ea6fe29f0316 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.237526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c67f4-d009-4852-89d2-ea6fe29f0316 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.237811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c67f4-d009-4852-89d2-ea6fe29f0316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.260427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63cce1-9265-4005-add1-c8c766ddf1e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a41869fa0bf49ea8f684f7585d96578, user_id=339d9b743e15476480900fdeb90f694f, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.264830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a63cce1-9265-4005-add1-c8c766ddf1e7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.270090 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d99c67f4-d009-4852-89d2-ea6fe29f0316 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:38.271365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d99c67f4-d009-4852-89d2-ea6fe29f0316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:38.280324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1193/2396] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:38.294752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91aec3-a918-4e49-a42e-4f8d78c2a87b None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.295099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91aec3-a918-4e49-a42e-4f8d78c2a87b None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.295884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91aec3-a918-4e49-a42e-4f8d78c2a87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.316766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5c91aec3-a918-4e49-a42e-4f8d78c2a87b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:38.317938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5c91aec3-a918-4e49-a42e-4f8d78c2a87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:38.326245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1194/2397] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:38.330757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91aec3-a918-4e49-a42e-4f8d78c2a87b None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.339835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c91aec3-a918-4e49-a42e-4f8d78c2a87b None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.344916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.346759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.346759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.346759 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.346992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:38.347258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.349008 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.363652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1195/2398] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:15:38.367999 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.379279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cbe4d7-c1a7-4ccd-8b81-6dcd95dfc929 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.379784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1204/2399] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:38 2022] PUT /identity/v3/projects/8a41869fa0bf49ea8f684f7585d96578/users/339d9b743e15476480900fdeb90f694f/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:38.391377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.391737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.392088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.531340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.532016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.532304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.532588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.533297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.534409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] MFA Rules not processed for user `339d9b743e15476480900fdeb90f694f`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:38.535412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.535721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:38.613756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1205/2400] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:38.666768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.668269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:38.678102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1196/2401] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:38 2022] POST /identity/v3/projects => generated 389 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:15:38.681789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.749750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09e7cb1-d241-4134-9c5d-3dc5ea372e23 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.772787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72ea52-b83a-4b74-9957-8312cd27d822 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.789318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.790196 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.790735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.791086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.792040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.794189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.813531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.814356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.814712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.815061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.872435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.873883 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.884358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1206/2402] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:15:38.910196 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.911608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:38.926963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b07ae3-15e3-46e4-be59-464f07d516b6 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.931701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.932540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:38.932657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:38.932897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] PATH_INFO: `/v3/users/339d9b743e15476480900fdeb90f694f` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:38.934191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=339d9b743e15476480900fdeb90f694f)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:38.936479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:38.975711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1197/2403] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:38 2022] POST /identity/v3/users => generated 295 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:38.976673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:38.994303 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68280-5ad7-4ff1-b5f8-fcacef0bb136 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:38.999075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:38.999743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:39.000030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:39.000309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:39.001040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:39.002671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:39.013110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1198/2404] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:39.017659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:39.027845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65104fbe-3011-4c56-afb5-f9c5efee983a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:39.034096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:39.034898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:39.035297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:39.035686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:39.036584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:39.040156 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:39.064405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1199/2405] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:39.068383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:39.079734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e0a829-ac23-44ce-8ad0-c2413ea439bf admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:39.086205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f0afe7-a8ee-46fb-846d-a31367d6bda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:39.087492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0afe7-a8ee-46fb-846d-a31367d6bda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:39.087780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0afe7-a8ee-46fb-846d-a31367d6bda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:39.088049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f0afe7-a8ee-46fb-846d-a31367d6bda8 admin admin] PATH_INFO: `/v3/projects/df69c0d8cd3248c5ade904bb3acf0491/users/a1b1e9942b924682b9296f9e6011c045/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:39.104716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0afe7-a8ee-46fb-846d-a31367d6bda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df69c0d8cd3248c5ade904bb3acf0491, user_id=a1b1e9942b924682b9296f9e6011c045, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:39.107454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f0afe7-a8ee-46fb-846d-a31367d6bda8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:39.236890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:39.246840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1200/2406] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:39 2022] PUT /identity/v3/projects/df69c0d8cd3248c5ade904bb3acf0491/users/a1b1e9942b924682b9296f9e6011c045/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:39.257745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:39.258037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:39.258342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:39.277635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '339d9b743e15476480900fdeb90f694f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:39.293874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '339d9b743e15476480900fdeb90f694f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:39.297650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '339d9b743e15476480900fdeb90f694f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:39.312537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] MFA Rules not processed for user `a1b1e9942b924682b9296f9e6011c045`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:39.317022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:39.328141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '339d9b743e15476480900fdeb90f694f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:39.399437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1201/2407] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:39.411268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:39.412700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:39.434240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-8be1425e-f8bd-45a9-b30a-236e17a71490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '339d9b743e15476480900fdeb90f694f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:39.441616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1207/2408] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:38 2022] DELETE /identity/v3/users/339d9b743e15476480900fdeb90f694f => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:15:39.450597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:39.451009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:39.451491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:39.488310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:39.492444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:39.495427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa3ba5-d454-495e-a566-fa0db5e285c1 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:39.571456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:39.572329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:39.572758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:39.573362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:39.575357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:39.593613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1208/2409] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:39.600564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:39.602185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:39.694958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c858-7d42-41a3-bbda-3d4ed5c5f4d5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:39.699123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00456cb-2e21-4a7c-af1b-05aa4486ee8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:39.700013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00456cb-2e21-4a7c-af1b-05aa4486ee8f service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:39.700504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00456cb-2e21-4a7c-af1b-05aa4486ee8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:39.700889 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00456cb-2e21-4a7c-af1b-05aa4486ee8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:39.707527 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:39.711599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:39.733577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00456cb-2e21-4a7c-af1b-05aa4486ee8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:39.741537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00456cb-2e21-4a7c-af1b-05aa4486ee8f service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:39.744918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1202/2410] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:15:39.775682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1209/2411] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:15:40.585318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:40.587210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:40.607365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67a53-1b8d-4562-b05e-b29d88633273 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:40.617250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:40.635581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:40.636763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:40.637068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:40.649521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] PATH_INFO: `/v3/projects/8a41869fa0bf49ea8f684f7585d96578` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:40.653594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a41869fa0bf49ea8f684f7585d96578)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:40.657721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:40.821903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:40.822358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:40.822598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:40.839514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a41869fa0bf49ea8f684f7585d96578', 'request_id': 'req-bce54560-8de0-42b0-8b5c-0e959402760e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:40.858695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] MFA Rules not processed for user `baac9b5e923747e6986720e3ac849056`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:40.859967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:40.934542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1210/2412] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:40.954586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a41869fa0bf49ea8f684f7585d96578', 'request_id': 'req-bce54560-8de0-42b0-8b5c-0e959402760e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:40.964359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a41869fa0bf49ea8f684f7585d96578', 'request_id': 'req-bce54560-8de0-42b0-8b5c-0e959402760e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:40.979220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] The token cache is being invalidate because project 8a41869fa0bf49ea8f684f7585d96578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:40.983140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:40.983140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bce54560-8de0-42b0-8b5c-0e959402760e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:40.988141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1203/2413] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:40 2022] DELETE /identity/v3/projects/8a41869fa0bf49ea8f684f7585d96578 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:41.463331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:42.028981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:42.446064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:43.628746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:43.663035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:45.314043 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:45.532094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:45.649038 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:45.650706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:45.713816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c0f4c-3f2d-4a71-97c4-2b8654e2bced None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:45.719401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:45.720295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:45.720584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:45.720914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] PATH_INFO: `/v3/users/d8ca99e72c0f4e48a38a84e4858dbdde` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:45.722146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8ca99e72c0f4e48a38a84e4858dbdde)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:45.724084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:45.945816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8ca99e72c0f4e48a38a84e4858dbdde'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:45.953546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8ca99e72c0f4e48a38a84e4858dbdde'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:45.954266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8ca99e72c0f4e48a38a84e4858dbdde'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:45.967773 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8ca99e72c0f4e48a38a84e4858dbdde'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:46.056212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8ca99e72c0f4e48a38a84e4858dbdde'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:46.064003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1211/2414] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:15:45 2022] DELETE /identity/v3/users/d8ca99e72c0f4e48a38a84e4858dbdde => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:46.068658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:46.068928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:46.069216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:46.107355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:46.109239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:46.172070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1204/2415] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:15:46.181725 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:46.183146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:46.242207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29f917-0907-498f-a2a8-3485984a54f8 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:46.245798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a02aa-fbc6-447a-a4a8-d2dbdb2034b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:46.246442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a02aa-fbc6-447a-a4a8-d2dbdb2034b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:46.246720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a02aa-fbc6-447a-a4a8-d2dbdb2034b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:46.246992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a02aa-fbc6-447a-a4a8-d2dbdb2034b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:46.257616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a02aa-fbc6-447a-a4a8-d2dbdb2034b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:46.259042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a02aa-fbc6-447a-a4a8-d2dbdb2034b2 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:46.269212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1212/2416] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:15:46.704240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:46.705785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:46.718182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155087f-d0c1-4439-b4ce-6222ee069035 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:46.722861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:46.723721 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:46.723998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:46.724293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] PATH_INFO: `/v3/projects/ef6cbf12b58d488b88c1ac4dd0d45712` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:46.725506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6cbf12b58d488b88c1ac4dd0d45712)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:46.727467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:46.835253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6cbf12b58d488b88c1ac4dd0d45712', 'request_id': 'req-dfe8d3f2-5762-477c-ae6e-68245790129e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:46.913653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6cbf12b58d488b88c1ac4dd0d45712', 'request_id': 'req-dfe8d3f2-5762-477c-ae6e-68245790129e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:46.919653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6cbf12b58d488b88c1ac4dd0d45712', 'request_id': 'req-dfe8d3f2-5762-477c-ae6e-68245790129e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:46.932152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] The token cache is being invalidate because project ef6cbf12b58d488b88c1ac4dd0d45712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:15:46.932438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:46.933057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dfe8d3f2-5762-477c-ae6e-68245790129e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:15:46.934669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1205/2417] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:15:46 2022] DELETE /identity/v3/projects/ef6cbf12b58d488b88c1ac4dd0d45712 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:46.940285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88f393-deec-492b-9711-245150b942fc None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:46.940748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88f393-deec-492b-9711-245150b942fc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:46.940858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88f393-deec-492b-9711-245150b942fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:46.959095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2a88f393-deec-492b-9711-245150b942fc None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:46.960283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2a88f393-deec-492b-9711-245150b942fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:47.020316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1213/2418] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:47.028901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace04e9-a290-41c6-9f7b-98dfa6fb76df None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.029808 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace04e9-a290-41c6-9f7b-98dfa6fb76df None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.030135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace04e9-a290-41c6-9f7b-98dfa6fb76df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.048685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-bace04e9-a290-41c6-9f7b-98dfa6fb76df None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:47.050368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bace04e9-a290-41c6-9f7b-98dfa6fb76df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:47.058284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1206/2419] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:47.062897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88f393-deec-492b-9711-245150b942fc None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:47.072723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88f393-deec-492b-9711-245150b942fc None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:47.077483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:47.078243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.078485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.078766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.079535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:15:47.079842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:47.081480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:47.093878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1214/2420] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:15:47.098034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace04e9-a290-41c6-9f7b-98dfa6fb76df None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:47.108234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace04e9-a290-41c6-9f7b-98dfa6fb76df None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:47.113185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:47.113876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.114157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.114433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.115140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:47.117078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:47.208928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:47.215942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1207/2421] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:15:47 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:15:47.220528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:47.230989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21747f05-cb66-44b4-9f33-3924c7ee09c0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:47.295220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:47.295967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.296369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.296904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.297870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:47.299884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:47.310409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:47.398190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1215/2422] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:15:47 2022] POST /identity/v3/users => generated 292 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:15:47.399628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:47.411621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8b562-f311-4573-9e97-6116d0a18505 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:47.416647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:47.417374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.417659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.417935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.418668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:47.420376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:47.432113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1208/2423] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:47.435826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:47.447346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74902f58-71a3-4d36-a52e-0b4f0ad3a77f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:47.453388 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472ef9-748e-44e4-baa2-c4a03f1409a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:47.454081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472ef9-748e-44e4-baa2-c4a03f1409a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.454369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472ef9-748e-44e4-baa2-c4a03f1409a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.454652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472ef9-748e-44e4-baa2-c4a03f1409a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.455487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25472ef9-748e-44e4-baa2-c4a03f1409a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:47.457342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25472ef9-748e-44e4-baa2-c4a03f1409a0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:47.470109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1216/2424] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:15:47.474413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:47.485339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5537480-f660-4cc2-b262-476060181ad7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:47.492225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:47.493247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.493600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.493972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] PATH_INFO: `/v3/projects/0f0441599dc5456da3e0f48be7dd2d20/users/bb2d87f281834e4697a8c3559b6e83a2/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.511928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0441599dc5456da3e0f48be7dd2d20, user_id=bb2d87f281834e4697a8c3559b6e83a2, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:47.515842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:47.600495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1209/2425] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:15:47 2022] PUT /identity/v3/projects/0f0441599dc5456da3e0f48be7dd2d20/users/bb2d87f281834e4697a8c3559b6e83a2/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:15:47.605584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212297b-9b79-4092-80f8-aaddb97a4d70 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.605882 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212297b-9b79-4092-80f8-aaddb97a4d70 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.606156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212297b-9b79-4092-80f8-aaddb97a4d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.624233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2212297b-9b79-4092-80f8-aaddb97a4d70 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:47.625445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2212297b-9b79-4092-80f8-aaddb97a4d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:47.683089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1217/2426] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:47.693097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:47.694792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:47.723659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:47.754191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0a24a-e798-436e-a259-e78c7bf206dd admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:47.758040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:47.758694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:47.758978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:47.759258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:47.768532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:47.770028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:47.780093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1210/2427] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:47.794439 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:49.483232 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53563118-8fcb-4604-9c86-7499d5705593 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:49.484096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53563118-8fcb-4604-9c86-7499d5705593 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:49.484096 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53563118-8fcb-4604-9c86-7499d5705593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:49.514644 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-53563118-8fcb-4604-9c86-7499d5705593 None None] MFA Rules not processed for user `bb2d87f281834e4697a8c3559b6e83a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:49.515886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-53563118-8fcb-4604-9c86-7499d5705593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:49.593420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1218/2428] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:49.600724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:49.602017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:15:49.603801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:49.684841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0fc39c-1308-4c27-b362-eb57d86eb5d9 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:15:49.689216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:15:49.692578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:49.692578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:49.692578 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:49.699309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:15:49.700782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:15:49.711919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1211/2429] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:15:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:15:51.142903 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:51.641036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:53.131491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:53.845720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:53.862787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeea4f-5377-4e9a-9e98-278c4e128f91 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:15:53.863074 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeea4f-5377-4e9a-9e98-278c4e128f91 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:15:53.863354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdeea4f-5377-4e9a-9e98-278c4e128f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:15:53.882493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ffdeea4f-5377-4e9a-9e98-278c4e128f91 None None] MFA Rules not processed for user `bb2d87f281834e4697a8c3559b6e83a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:15:53.883744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ffdeea4f-5377-4e9a-9e98-278c4e128f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:15:53.896015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1219/2430] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:15:56.172984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:56.525407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:57.331098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:57.519695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:58.160414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:58.211086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:59.251031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:59.291288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:15:59.982342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.005511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.153984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.308156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.640945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.697016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.816666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.953753 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:00.976602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.051149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.074415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.102367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.268767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.302418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.462182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.490784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.616433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.639481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.714005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.737234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.839495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.868038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.954023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:01.982334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.113857 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.142623 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.279368 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.308239 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.508354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.582398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.816874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:02.919783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:03.136786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:03.404597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:03.713551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:03.976882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.004619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.279778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.296841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.393365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=23,keystone:SELECT=74,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.410605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.513829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.536298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.628633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.656594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.770825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.793700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.867767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.890608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.959143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:04.982015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.073646 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.097679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.206137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.228331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.256691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.405023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.433591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:05.879694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:06.165449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:07.525413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:07.604704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:09.005621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:09.103996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:09.347927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:09.731017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:09.961315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:10.170821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:10.190018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:10.209535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:10.211269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:10.226981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0b3666-e3e5-4603-8f42-e7f38a0d3ba7 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:10.231707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:10.232838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:10.233124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:10.233439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] PATH_INFO: `/v3/users/82803409c2344e379035e0fb3ee6b023` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:10.234642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82803409c2344e379035e0fb3ee6b023)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:10.237399 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:10.676236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82803409c2344e379035e0fb3ee6b023'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:10.677512 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82803409c2344e379035e0fb3ee6b023'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:10.685841 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82803409c2344e379035e0fb3ee6b023'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:10.687092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82803409c2344e379035e0fb3ee6b023'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:10.701674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82803409c2344e379035e0fb3ee6b023'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:10.816330 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1212/2431] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:16:10 2022] DELETE /identity/v3/users/82803409c2344e379035e0fb3ee6b023 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:10.818295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:10.818590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:10.818901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:10.849919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:10.851832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:10.891603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:10.929733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1220/2432] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:10.939752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:10.941248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:10.968108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:11.009555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cc8c7-9043-4009-ae4b-a8e896909ea4 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:11.013485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:11.014145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:11.014430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:11.015477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:11.025561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:11.027001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:11.038163 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1213/2433] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:16:11.108019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:11.531929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:11.534817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:11.552885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6caa0-1b79-461f-91b7-18e796f33891 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:11.561503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:11.574409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:11.574409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:11.574409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] PATH_INFO: `/v3/projects/cd01cba058564916a3422acc5bbec080` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:11.574409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd01cba058564916a3422acc5bbec080)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:11.574409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:11.627563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:11.687185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd01cba058564916a3422acc5bbec080', 'request_id': 'req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:11.747948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd01cba058564916a3422acc5bbec080', 'request_id': 'req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:11.763409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd01cba058564916a3422acc5bbec080', 'request_id': 'req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:11.778221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] The token cache is being invalidate because project cd01cba058564916a3422acc5bbec080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:16:11.779124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:11.780803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a2c69eb3-5ba8-44ea-ab1c-a2b3c3d38c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:11.799255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1221/2434] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:16:11 2022] DELETE /identity/v3/projects/cd01cba058564916a3422acc5bbec080 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:16:12.240017 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:12.634218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:12.653632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:12.655251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:12.665772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:12.671333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b074b2f-9faf-4dc3-95eb-3f10ecae34b6 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:12.750876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:12.752301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:12.752834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:12.753290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] PATH_INFO: `/v3/users/a1b1e9942b924682b9296f9e6011c045` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:12.755427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b1e9942b924682b9296f9e6011c045)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:12.757835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:13.014738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b1e9942b924682b9296f9e6011c045'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:13.015997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b1e9942b924682b9296f9e6011c045'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:13.025151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b1e9942b924682b9296f9e6011c045'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:13.026129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b1e9942b924682b9296f9e6011c045'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:13.041598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b1e9942b924682b9296f9e6011c045'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:13.179639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1214/2435] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:16:12 2022] DELETE /identity/v3/users/a1b1e9942b924682b9296f9e6011c045 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:13.184016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:13.184525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:13.185026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:13.210093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:13.211317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:13.302985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1222/2436] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:13.313318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:13.315414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:13.450864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:13.457281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eb0dc-f936-4d0d-b3c3-755f7e32cfa1 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:13.463867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53acf90c-de84-4280-984a-695dd8f07a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:13.464748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf90c-de84-4280-984a-695dd8f07a2c service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:13.465852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf90c-de84-4280-984a-695dd8f07a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:13.466192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53acf90c-de84-4280-984a-695dd8f07a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:13.483146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acf90c-de84-4280-984a-695dd8f07a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:13.489754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53acf90c-de84-4280-984a-695dd8f07a2c service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:13.510044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1215/2437] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:16:13.942204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:13.980358 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:13.982601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:14.004212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a03e7-af48-4633-8f92-21dca5fbc771 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:14.012902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:14.013918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.014205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.014480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] PATH_INFO: `/v3/projects/df69c0d8cd3248c5ade904bb3acf0491` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.017812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df69c0d8cd3248c5ade904bb3acf0491)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:14.018730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:14.143795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df69c0d8cd3248c5ade904bb3acf0491', 'request_id': 'req-7c7f2997-f7fe-4684-9771-729793ff4244'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:14.253647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df69c0d8cd3248c5ade904bb3acf0491', 'request_id': 'req-7c7f2997-f7fe-4684-9771-729793ff4244'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:14.268460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df69c0d8cd3248c5ade904bb3acf0491', 'request_id': 'req-7c7f2997-f7fe-4684-9771-729793ff4244'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:14.288387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] The token cache is being invalidate because project df69c0d8cd3248c5ade904bb3acf0491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:16:14.288387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:14.288387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7c7f2997-f7fe-4684-9771-729793ff4244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:14.288387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1223/2438] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:16:13 2022] DELETE /identity/v3/projects/df69c0d8cd3248c5ade904bb3acf0491 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:14.292455 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f620-e2ab-4479-adfe-6c3f3589aefb None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.292938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f620-e2ab-4479-adfe-6c3f3589aefb None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.293069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec2f620-e2ab-4479-adfe-6c3f3589aefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.316214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-6ec2f620-e2ab-4479-adfe-6c3f3589aefb None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:14.318818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6ec2f620-e2ab-4479-adfe-6c3f3589aefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:14.342334 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:14.346206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:14.389439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1216/2439] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:16:14.397784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1d660-4ddc-489c-af30-30e392056dce None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.398078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1d660-4ddc-489c-af30-30e392056dce None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.398361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f1d660-4ddc-489c-af30-30e392056dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.420211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-68f1d660-4ddc-489c-af30-30e392056dce None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:14.420211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-68f1d660-4ddc-489c-af30-30e392056dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:14.428828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1224/2440] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:14.433516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f620-e2ab-4479-adfe-6c3f3589aefb None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:14.443609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec2f620-e2ab-4479-adfe-6c3f3589aefb None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:14.450609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:14.451281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.451562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.451833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.452564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:16:14.452870 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:14.454709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:14.467025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1217/2441] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 24 10:16:14.473851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1d660-4ddc-489c-af30-30e392056dce None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:14.491129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f1d660-4ddc-489c-af30-30e392056dce None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:14.497261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:14.497488 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.497739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.498019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.498736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:14.500793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:14.647117 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1225/2442] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:16:14 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:16:14.648884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:14.660990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ffb71-be1c-4904-a97a-8838f4f5582a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:14.763072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:14.763776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.764063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.764345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.765068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:14.776780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:14.908686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1218/2443] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:16:14 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:16:14.912667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:14.923763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36789c92-2c8c-459b-a6c1-ee89ac386350 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:14.928609 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:14.929349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.929634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.929915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.930643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:14.932252 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:14.945409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1226/2444] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:16:14.949488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:14.961399 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7cedca-6793-421e-967e-1e39de7e8cdc admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:14.966201 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f97c0-aa05-4914-9c04-b37d3f6e2c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:14.966863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f97c0-aa05-4914-9c04-b37d3f6e2c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:14.967219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f97c0-aa05-4914-9c04-b37d3f6e2c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:14.967495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f97c0-aa05-4914-9c04-b37d3f6e2c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:14.968225 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f97c0-aa05-4914-9c04-b37d3f6e2c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:14.969911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f97c0-aa05-4914-9c04-b37d3f6e2c58 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:14.986109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1219/2445] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:16:14.987651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:14.998336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8850b5-3051-4133-9db4-8067aecbf5f7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:15.008735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:15.008735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:15.008735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:15.008735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] PATH_INFO: `/v3/projects/2606d8cef9c442c896c12b401c92326f/users/77f178a1600c4704a9f7dba7e3b8f36a/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:15.030134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2606d8cef9c442c896c12b401c92326f, user_id=77f178a1600c4704a9f7dba7e3b8f36a, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:15.033970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:15.118898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1227/2446] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:16:14 2022] PUT /identity/v3/projects/2606d8cef9c442c896c12b401c92326f/users/77f178a1600c4704a9f7dba7e3b8f36a/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:15.123941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a4ee6-1363-467e-a26c-1ed281cdf9c0 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:15.124304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a4ee6-1363-467e-a26c-1ed281cdf9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:15.125071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a4ee6-1363-467e-a26c-1ed281cdf9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:15.146615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b43a4ee6-1363-467e-a26c-1ed281cdf9c0 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:15.148170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b43a4ee6-1363-467e-a26c-1ed281cdf9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:15.224598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1220/2447] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:16:15.234127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:15.235921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:15.292399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc56ec-c34f-47ae-b399-ff53bbfb1d6f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:15.296073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:15.296970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:15.297247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:15.297549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:15.306497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:15.307925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:15.318396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1228/2448] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:16:15.319875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:19.183775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0774953-cf63-48eb-8696-5883e0ba5099 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:19.184720 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0774953-cf63-48eb-8696-5883e0ba5099 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:19.185189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0774953-cf63-48eb-8696-5883e0ba5099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:19.240626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:19.279218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c0774953-cf63-48eb-8696-5883e0ba5099 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:19.281539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c0774953-cf63-48eb-8696-5883e0ba5099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:19.373037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1221/2449] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:16:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:19.433728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:19.435255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:19.541728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c611850-eb49-44a4-bf37-4866e5c5c871 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:19.548353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:19.549079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:19.550645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:19.551076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:19.561109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:19.562804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:19.578024 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1229/2450] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:16:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:16:21.998167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe1739-80eb-4551-9761-2321f8744800 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:21.999246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe1739-80eb-4551-9761-2321f8744800 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:21.999706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efe1739-80eb-4551-9761-2321f8744800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:22.047948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-8efe1739-80eb-4551-9761-2321f8744800 None None] MFA Rules not processed for user `77f178a1600c4704a9f7dba7e3b8f36a`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:22.049415 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8efe1739-80eb-4551-9761-2321f8744800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:22.198626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:22.211475 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1222/2451] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:22.234483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d49aa-495c-4c8c-9949-39ada3664677 service swift] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:22.272626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:22.273313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:22.273790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:22.274231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:22.275109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:22.279565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:22.766907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1230/2452] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:16:22 2022] POST /identity/v3/projects => generated 385 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:16:22.767734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe1739-80eb-4551-9761-2321f8744800 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:22.787343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efe1739-80eb-4551-9761-2321f8744800 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:22.930111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:22.931632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:22.932562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:22.932989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:22.934522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:22.937802 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:23.081089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1223/2453] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:16:22 2022] POST /identity/v3/users => generated 299 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:16:23.085150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:23.097351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac7540-8c69-4db4-9fb3-e628e282919b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:23.113151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:23.115484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:23.116793 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:23.119665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:23.123655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:23.123655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:23.140013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1231/2454] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:16:23.144316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:23.161442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2ee01-3d68-4dae-ba81-b8b60f9704d7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:23.170076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:23.171139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:23.171744 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:23.172100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] PATH_INFO: `/v3/projects/55c5945038624ca4ab268e5587bc002d/users/58832bf1d66342d7bad90d67f9272e4f/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:23.176417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:23.192923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c5945038624ca4ab268e5587bc002d, user_id=58832bf1d66342d7bad90d67f9272e4f, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:23.196162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:23.314071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1224/2455] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:16:23 2022] PUT /identity/v3/projects/55c5945038624ca4ab268e5587bc002d/users/58832bf1d66342d7bad90d67f9272e4f/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:23.318698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:23.336678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a17c2d9-11ac-4112-8114-1065e64d6e2b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:23.425176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:23.426356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:23.426633 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:23.427079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:23.427956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:23.429675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:23.439545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:23.456659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:23.469384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1232/2456] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:16:23.479607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4451-7411-4241-86f7-1dc63e317637 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:23.484600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:23.488096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:23.488392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:23.488672 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] PATH_INFO: `/v3/projects/55c5945038624ca4ab268e5587bc002d/users/58832bf1d66342d7bad90d67f9272e4f/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:23.490918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c5945038624ca4ab268e5587bc002d, user_id=58832bf1d66342d7bad90d67f9272e4f, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:23.493612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:23.507354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1225/2457] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:16:23 2022] PUT /identity/v3/projects/55c5945038624ca4ab268e5587bc002d/users/58832bf1d66342d7bad90d67f9272e4f/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:30.926865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:30.929616 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:31.095122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1469df-af13-41d1-b8d7-47b94087fa0b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:31.179548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c1af0-e791-4942-9f31-f1b02d4e1ab9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:31.180906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c1af0-e791-4942-9f31-f1b02d4e1ab9 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:31.181518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c1af0-e791-4942-9f31-f1b02d4e1ab9 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:31.181992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c1af0-e791-4942-9f31-f1b02d4e1ab9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:31.184128 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f76c1af0-e791-4942-9f31-f1b02d4e1ab9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:31.197675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c1af0-e791-4942-9f31-f1b02d4e1ab9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:31.199534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c1af0-e791-4942-9f31-f1b02d4e1ab9 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:31.211645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1233/2458] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:16:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:16:32.931504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:32.933428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:33.075820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9306-d626-470a-b366-4e2868ccfe28 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:33.082147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:33.083121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:33.083410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:33.084296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:33.086289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:33.156490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888747-c2c8-4584-8a9e-44bd3725c8db None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:33.157608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888747-c2c8-4584-8a9e-44bd3725c8db None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:33.157608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c888747-c2c8-4584-8a9e-44bd3725c8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:33.191404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4c888747-c2c8-4584-8a9e-44bd3725c8db None None] MFA Rules not processed for user `58832bf1d66342d7bad90d67f9272e4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:33.192943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4c888747-c2c8-4584-8a9e-44bd3725c8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:33.246334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:33.248436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:33.272502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1226/2459] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:16:32 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:16:33.318086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:33.346589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1234/2460] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:33 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:16:38.825017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:38.826541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:38.914020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd93b-9011-491e-bd8a-2bb6697629ea service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:38.917834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:38.918509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:38.918803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:38.919081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:38.933640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:38.988259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:38.989951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:39.001332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1227/2461] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:16:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:16:39.363869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd707e-5a2e-4b18-a123-2dcad97ae5c3 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:39.366420 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd707e-5a2e-4b18-a123-2dcad97ae5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:39.367186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd707e-5a2e-4b18-a123-2dcad97ae5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:39.405106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-95fd707e-5a2e-4b18-a123-2dcad97ae5c3 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:39.406397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-95fd707e-5a2e-4b18-a123-2dcad97ae5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:39.477247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1235/2462] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:16:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:39.484487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:39.578858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf737c-d165-4e8f-ae3f-1508ccb50bde service glance] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:39.584311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:39.584971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:39.585306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:39.585661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:39.593875 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:39.595612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:39.603881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1228/2463] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:16:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:16:45.762786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:45.763343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:45.763343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:45.789910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:45.791447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:45.805719 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1236/2464] 10.222.0.35 () {58 vars in 967 bytes} [Tue May 24 10:16:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:45.823499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:45.834997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a959206-afd9-4e2d-8f19-a56190e9b400 service swift] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:45.841382 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:45.842147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:45.842489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:45.842846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:45.853154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:45.854790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:45.867810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1229/2465] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:16:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:16:50.098070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:50.099565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:50.181474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914d18-fdbc-4d08-9af0-4246474866b8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:50.186826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:50.187959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:50.188438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:50.188727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] PATH_INFO: `/v3/users/6e8f1cb9760f41c6b7c1eb379daf1b64` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:50.190124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e8f1cb9760f41c6b7c1eb379daf1b64)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:50.192146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:50.379849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e8f1cb9760f41c6b7c1eb379daf1b64'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:50.388624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e8f1cb9760f41c6b7c1eb379daf1b64'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:50.389359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e8f1cb9760f41c6b7c1eb379daf1b64'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:50.405217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e8f1cb9760f41c6b7c1eb379daf1b64'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:50.495013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-94246cef-9156-46d5-bcbe-401ecff8f57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e8f1cb9760f41c6b7c1eb379daf1b64'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:50.505148 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1237/2466] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:16:50 2022] DELETE /identity/v3/users/6e8f1cb9760f41c6b7c1eb379daf1b64 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:50.817513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:50.830589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a4472-2e33-43db-916a-a43a7f9768c1 service swift] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:50.890626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:50.891594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:50.891906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:50.892184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] PATH_INFO: `/v3/projects/a805d83a8e7747528285d543eb274ab6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:50.893516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a805d83a8e7747528285d543eb274ab6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:50.895427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:51.021716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:51.033247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a805d83a8e7747528285d543eb274ab6', 'request_id': 'req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:51.158624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a805d83a8e7747528285d543eb274ab6', 'request_id': 'req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:51.167588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a805d83a8e7747528285d543eb274ab6', 'request_id': 'req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:51.187722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] The token cache is being invalidate because project a805d83a8e7747528285d543eb274ab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:16:51.187994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:51.188663 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-0b154c1b-5721-4f7f-ac25-0e955b0e5c85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:16:51.190831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1230/2467] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:16:50 2022] DELETE /identity/v3/projects/a805d83a8e7747528285d543eb274ab6 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:16:51.196693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbef310-9004-467c-915e-236cc1d76df4 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.196693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbef310-9004-467c-915e-236cc1d76df4 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.197377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbef310-9004-467c-915e-236cc1d76df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.222209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-abbef310-9004-467c-915e-236cc1d76df4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:51.223419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-abbef310-9004-467c-915e-236cc1d76df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:51.229053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:51.287370 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1238/2468] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:51.295115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837726b9-85c1-4f8d-8eb6-640f6ef29ae2 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.295406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837726b9-85c1-4f8d-8eb6-640f6ef29ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.295685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837726b9-85c1-4f8d-8eb6-640f6ef29ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.316111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-837726b9-85c1-4f8d-8eb6-640f6ef29ae2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:51.317344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-837726b9-85c1-4f8d-8eb6-640f6ef29ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:51.325664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1231/2469] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:51.339510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbef310-9004-467c-915e-236cc1d76df4 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:51.350562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbef310-9004-467c-915e-236cc1d76df4 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:51.355221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e3a866-5657-4dca-acbd-f47cd4c9189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:51.360269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3a866-5657-4dca-acbd-f47cd4c9189a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.360269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3a866-5657-4dca-acbd-f47cd4c9189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.360269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e3a866-5657-4dca-acbd-f47cd4c9189a admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.360269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3a866-5657-4dca-acbd-f47cd4c9189a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:16:51.360269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3a866-5657-4dca-acbd-f47cd4c9189a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:51.360269 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e3a866-5657-4dca-acbd-f47cd4c9189a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:51.371248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1239/2470] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:16:51.376685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837726b9-85c1-4f8d-8eb6-640f6ef29ae2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:51.386095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837726b9-85c1-4f8d-8eb6-640f6ef29ae2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:51.390863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:51.391606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.391895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.392174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.392904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:51.394761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952fd8f-27c4-484b-a8d1-eb86623ff847 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.395185 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952fd8f-27c4-484b-a8d1-eb86623ff847 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.395360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:51.395476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952fd8f-27c4-484b-a8d1-eb86623ff847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.417322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-b952fd8f-27c4-484b-a8d1-eb86623ff847 None None] MFA Rules not processed for user `77f178a1600c4704a9f7dba7e3b8f36a`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:51.418558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b952fd8f-27c4-484b-a8d1-eb86623ff847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:51.477176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1240/2471] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:16:51.479515 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1232/2472] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:16:51 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:16:51.483895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952fd8f-27c4-484b-a8d1-eb86623ff847 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:51.495375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952fd8f-27c4-484b-a8d1-eb86623ff847 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:51.563472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:51.564093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.564378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.564654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.565401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:51.567301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:51.655898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1241/2473] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:16:51 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:16:51.660997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:51.671835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b060280-fc95-4fdf-aa64-9b9abe78d4b7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:51.676943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:51.677648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.677933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.678210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.678971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:51.680579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:51.691522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1233/2474] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:16:51.695897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:51.706832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d024f33c-23a3-409e-a837-3225828902ca admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:51.711547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d4eb92-6d37-4fd5-b583-e6074e58663c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:51.714723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4eb92-6d37-4fd5-b583-e6074e58663c admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.714723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4eb92-6d37-4fd5-b583-e6074e58663c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.714723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d4eb92-6d37-4fd5-b583-e6074e58663c admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.714723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4eb92-6d37-4fd5-b583-e6074e58663c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:51.715203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d4eb92-6d37-4fd5-b583-e6074e58663c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:51.726533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1242/2475] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:16:51.730827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:51.741542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfb235-e801-40b2-95cd-e279d482baca admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:51.746468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:51.749212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.749212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.749212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] PATH_INFO: `/v3/projects/28478e7f51704defa09c87ffc818cd18/users/e8f1082f78d64bb69ed4c8e5cd2e75dc/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.763762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28478e7f51704defa09c87ffc818cd18, user_id=e8f1082f78d64bb69ed4c8e5cd2e75dc, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:51.767253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:51.879541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1234/2476] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:16:51 2022] PUT /identity/v3/projects/28478e7f51704defa09c87ffc818cd18/users/e8f1082f78d64bb69ed4c8e5cd2e75dc/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:16:51.889546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982247dd-635f-4553-9ad0-0faf68a6c2ea None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:51.889835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982247dd-635f-4553-9ad0-0faf68a6c2ea None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:51.890108 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982247dd-635f-4553-9ad0-0faf68a6c2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:51.919426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-982247dd-635f-4553-9ad0-0faf68a6c2ea None None] MFA Rules not processed for user `e8f1082f78d64bb69ed4c8e5cd2e75dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:51.920552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-982247dd-635f-4553-9ad0-0faf68a6c2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:51.932608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:51.990645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1243/2477] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:16:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:52.002385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:52.004280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:52.063784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d7393-6db2-48d9-b1ac-0d3adc8e5d47 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:52.067619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:52.068276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:52.068562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:52.068846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:52.077789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:52.079544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:52.088621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1235/2478] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:16:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:16:55.158109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dd22c-f3f1-469f-b4cf-c6bcc49d70ee None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:55.158639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dd22c-f3f1-469f-b4cf-c6bcc49d70ee None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:55.158749 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3dd22c-f3f1-469f-b4cf-c6bcc49d70ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:55.177612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6b3dd22c-f3f1-469f-b4cf-c6bcc49d70ee None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:16:55.178828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6b3dd22c-f3f1-469f-b4cf-c6bcc49d70ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:55.236310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1244/2479] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:16:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:16:55.248186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:16:55.249747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:16:55.324736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:16:55.337806 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15051465-db34-4314-b44e-8cc6b5c7d03d service glance] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:16:55.342956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f8705-dde1-4117-ae46-f6471e9b2369 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:16:55.344783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f8705-dde1-4117-ae46-f6471e9b2369 service swift] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:16:55.344783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f8705-dde1-4117-ae46-f6471e9b2369 service swift] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:16:55.344985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f8705-dde1-4117-ae46-f6471e9b2369 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:16:55.353970 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f8705-dde1-4117-ae46-f6471e9b2369 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:16:55.355643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f8705-dde1-4117-ae46-f6471e9b2369 service swift] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:16:55.368632 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1236/2480] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:16:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:17:05.229245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=47 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:17:05.512128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:04.695298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:18:04.695842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:18:04.695842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:18:04.715161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] MFA Rules not processed for user `7bffd469507b4dbc8d3146f051c8c8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:18:04.716344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:04.770723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1245/2481] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:18:14.763266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=25 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:15.935373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:18:15.935373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:18:15.935373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:18:15.959229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] MFA Rules not processed for user `baac9b5e923747e6986720e3ac849056`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:18:15.960673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:16.021504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1237/2482] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:18:26.013751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=25 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:18:45.579563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:18:45.580984 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:45.640312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c30cf-e6a9-4eb7-a054-d050bfa4831d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:18:45.643943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:18:45.644673 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:18:45.644957 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:18:45.645281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:18:45.648070 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:45.731429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:18:45.732907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:18:45.755924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1246/2483] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:18:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:18:46.940270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:18:46.941840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:46.954219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ec7e3-c863-4f7d-a4a5-ff38ad9b80f3 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:18:46.957823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:18:46.958489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:18:46.958881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:18:46.959242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:18:46.960887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:46.973660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:18:46.975457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:18:46.986234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1238/2484] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:18:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:18:53.548527 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:18:53.549932 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:53.606001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a24e6dd-272e-4b28-879e-71ff7301d459 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:18:53.609547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:18:53.610188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:18:53.610467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:18:53.610743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:18:53.612497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:53.623397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:18:53.624956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:18:53.633811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1247/2485] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:18:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:18:54.865884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:18:54.867461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:54.882458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4fe8-e38a-448a-89b2-586ae5cadd30 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:18:54.888350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:18:54.889433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:18:54.889433 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:18:54.889828 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:18:54.891646 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:54.904256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:18:54.905888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:18:54.915660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1239/2486] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:18:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:18:55.821317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:18:55.830459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4415d921-f0b6-4a78-bce2-35dee3591879 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:18:55.834428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:18:55.835053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:18:55.835332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:18:55.835602 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:18:55.837369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:18:55.846789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:18:55.848213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:18:55.856817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1248/2487] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:18:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:19:02.758435 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:19:02.769561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1262f9-cb08-4683-a9fe-4bd7bf41f4ab service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:19:02.775880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:19:02.776525 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:19:02.776834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:19:02.777070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:19:02.778858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:19:02.791876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:19:02.793465 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:19:02.804292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1240/2488] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:19:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:19:05.853281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:19:12.799745 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=18 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:25.097755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:20:25.111026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1cc7f8-8e3f-4a79-970e-c52de97aa9a5 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:20:25.116424 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:20:25.117054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:20:25.117477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:20:25.117759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:20:25.119486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:20:25.178233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:20:25.179893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:20:25.189926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1249/2489] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:20:25 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:20:35.184732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=28 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:20:54.965582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:20:54.966593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:20:54.978197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3a2a9-26ab-40a5-be19-ae6d594ffa73 service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:20:54.981741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:20:54.982371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:20:54.982656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:20:54.982933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:20:54.984621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:20:55.044158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:20:55.045612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:20:55.055386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1241/2490] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:20:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:21:05.050575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=28 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:21:34.226025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:21:34.227454 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:21:34.300782 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4d92f-42a9-43b3-b314-2c219899ade7 service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:21:34.306812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:21:34.307468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:21:34.307751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:21:34.308028 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:21:34.309823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:21:34.321717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:21:34.323315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:21:34.333529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1250/2491] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:21:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:21:44.329001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=32 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:22:55.368975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:22:55.377573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beb9723-d4b8-4924-a390-22214d117caa service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:22:55.381107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:22:55.381773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:22:55.382059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:22:55.382338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:22:55.390629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:22:55.392097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:22:55.400838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1242/2492] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:22:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:22:55.481075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:22:55.491200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe952f2e-b45f-4f28-8bfd-a5c70f51d9fc service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:22:55.494817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:22:55.495456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:22:55.495763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:22:55.496057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:22:55.497887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:22:55.557021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:22:55.558500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:22:55.567620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1251/2493] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:22:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:23:05.396822 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:05.563494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=28 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:23:47.498333 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:23:47.507926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96d6d-174f-46fa-9497-32f2fbe93653 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:23:47.511449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:23:47.512076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:23:47.512355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:23:47.512648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:23:47.523844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:23:47.525452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:23:47.537483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1243/2494] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:23:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:23:53.911236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:23:53.923431 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57758ff5-bd88-4242-b4a9-eef933efa6d9 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:23:53.929387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:23:53.930029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:23:53.930310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:23:53.930592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:23:53.938479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:23:53.940050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:23:53.967043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1252/2495] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:23:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:23:57.531366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:03.964409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=13 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:31.870192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:24:31.878793 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746acd8c-2031-46dc-8be9-8faf6d4e5cc7 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:24:31.882976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:24:31.883605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:24:31.883890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:24:31.884166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:24:31.894291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:24:31.896542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:24:31.907467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1244/2496] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:24:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:24:38.884473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:24:38.894006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc463bb-88fd-4c27-ad09-e05051ccc94e service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:24:38.898515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:24:38.899163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:24:38.899471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:24:38.899786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:24:38.907481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:24:38.908907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:24:38.917842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1253/2497] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:24:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:24:41.902630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:46.956638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:24:46.959437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:24:46.975946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b50fb1-30db-43ae-afc6-913730313954 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:24:46.981949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:24:46.982776 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:24:46.983015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:24:46.983565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:24:46.986783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:24:46.998095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:24:46.999673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:24:47.010917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1245/2498] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:24:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:24:48.914093 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:55.821997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:24:55.831164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6d8ce-cd70-4c80-b43f-7f5cefa61335 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:24:55.835293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:24:55.835924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:24:55.836205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:24:55.836482 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:24:55.844802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:24:55.846265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:24:55.873305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1254/2499] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:24:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:24:57.005961 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:24:57.822152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:24:57.833600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea3343-dfa8-45e1-95b0-96de5dc0b0bb service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:24:57.840986 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:24:57.841948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:24:57.842236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:24:57.842545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:24:57.844510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:24:57.857682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:24:57.859366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:24:57.871392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1246/2500] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:24:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:25:05.870700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=13 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:07.866773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:26.076247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:25:26.123246 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfbade-255f-452e-8bf5-393728826488 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:25:26.128785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:25:26.129461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:25:26.129746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:25:26.130027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:25:26.137823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:25:26.139415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:25:26.167732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1255/2501] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:25:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:25:36.164395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=27 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:25:56.048500 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:25:56.090700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5412b-422e-4ea6-8030-3006113e287a service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:25:56.096844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:25:56.097508 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:25:56.097790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:25:56.098068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:25:56.105881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:25:56.107457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:25:56.134703 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1247/2502] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:25:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:26:06.132137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=27 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:26:42.221386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:26:42.251631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b46c1-3f8d-434b-97d0-6b90c7ec62ae service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:26:42.258835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:26:42.259470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:26:42.259748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:26:42.260067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:26:42.268759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:26:42.270365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:26:42.280034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1256/2503] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:26:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:26:52.275791 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=13 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:28:56.784306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:28:56.852497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a920a7c-baa3-45c7-b67e-f498f2b80dc8 service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:28:56.856199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:28:56.863492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:28:56.863492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:28:56.863492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:28:56.866735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:28:56.868349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:28:56.895967 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1248/2504] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:28:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:28:56.942255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:28:56.951406 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a68c5c-a47d-48cd-862c-51ae38a8309f service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:28:56.956336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:28:56.957073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:28:56.957473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:28:56.957756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:28:56.959762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:28:57.047098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:28:57.048627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:28:57.067020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1257/2505] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:28:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:28:57.110807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:28:57.119334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe429a4b-3bdf-4114-bb63-cc1ee4b4cba5 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:28:57.122944 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:28:57.123607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:28:57.123892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:28:57.124171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:28:57.126716 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:28:57.202649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:28:57.204758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:28:57.214158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1249/2506] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:28:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:28:57.263707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:28:57.274241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37181b57-8776-4fd4-8707-4ca83f8b8e8d service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:28:57.277970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:28:57.278612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:28:57.278897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:28:57.279173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:28:57.281008 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:28:57.371055 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:28:57.372588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:28:57.393751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1258/2507] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:28:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:29:07.209833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=70 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:07.390920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:32.756323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:29:32.758039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:29:32.835537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7baba9b-6e78-44fe-b7c3-d8de1f9e5faf service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:29:32.839076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:29:32.839718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:29:32.839995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:29:32.840266 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:29:32.842276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:29:32.855741 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:29:32.857169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:29:32.868257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1250/2508] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:29:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:29:42.863167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=35 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:29:45.655265 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:29:45.656896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:29:45.668728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973a9e2-2bbc-4d56-ae8b-1a190b3ebb3c service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:29:45.672381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:29:45.673018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:29:45.673327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:29:45.673605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:29:45.675391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:29:45.685320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:29:45.686748 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:29:45.694911 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1259/2509] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:29:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:29:55.691079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:26.604016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:26.616104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c845e3c-0a24-4f0f-8181-4c096b6006b4 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:26.622047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:26.622691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:26.622972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:26.623248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:26.625103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:26.695496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:26.697068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:26.705919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1251/2510] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:30:26 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:30:36.701770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=33 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:48.131211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:48.140963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c9631-342d-42ca-8602-fc4728d8b6e7 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:48.144861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:48.145554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:48.145864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:48.146145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:48.147966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:48.232344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:48.233878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:48.253044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1260/2511] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:30:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:30:51.702377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:51.703785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:51.730696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb6668-64cf-48a4-83d5-73ff98d21a61 service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:51.735765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:51.736422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:51.736708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:51.736991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:51.738771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:51.751113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:51.752688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:51.763334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1252/2512] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:30:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:30:55.937280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:55.938742 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:55.954171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6489a2-345f-4667-97f1-46674e8d7696 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:55.956347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:55.957217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:55.957528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:55.957804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] PATH_INFO: `/v3/users/577c7840904d4f2180f7e5bd0f530a9a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:55.958991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=577c7840904d4f2180f7e5bd0f530a9a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:55.960926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:56.127229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '577c7840904d4f2180f7e5bd0f530a9a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.134450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '577c7840904d4f2180f7e5bd0f530a9a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.135777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '577c7840904d4f2180f7e5bd0f530a9a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.148073 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '577c7840904d4f2180f7e5bd0f530a9a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.199481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '577c7840904d4f2180f7e5bd0f530a9a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.205785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1261/2513] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:30:55 2022] DELETE /identity/v3/users/577c7840904d4f2180f7e5bd0f530a9a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:30:56.446314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:56.460733 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8cc1ec-c73c-4db7-9195-8404c4bfe41b service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:56.516625 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:56.517528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:56.517818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:56.518097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] PATH_INFO: `/v3/projects/5bf4c198860648cd874233ba27d1b971` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:56.519281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf4c198860648cd874233ba27d1b971)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:56.521215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:56.628756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf4c198860648cd874233ba27d1b971', 'request_id': 'req-d307320c-c4be-45a7-931b-8d7121dd9bcc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.713791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf4c198860648cd874233ba27d1b971', 'request_id': 'req-d307320c-c4be-45a7-931b-8d7121dd9bcc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.719567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf4c198860648cd874233ba27d1b971', 'request_id': 'req-d307320c-c4be-45a7-931b-8d7121dd9bcc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.731388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] The token cache is being invalidate because project 5bf4c198860648cd874233ba27d1b971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:30:56.731678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.732320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:56.733892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1253/2514] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:30:56 2022] DELETE /identity/v3/projects/5bf4c198860648cd874233ba27d1b971 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:30:56.738112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:56.739535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:56.753155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c6f8a-84f9-4c64-9de0-006a36666c16 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:56.785448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:56.807939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:56.808811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:56.809087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:56.809386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] PATH_INFO: `/v3/users/77fc5b985da047a7aec85bc3aa4f1957` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:56.822718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77fc5b985da047a7aec85bc3aa4f1957)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:56.824519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:57.106497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77fc5b985da047a7aec85bc3aa4f1957'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.113461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77fc5b985da047a7aec85bc3aa4f1957'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.114136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77fc5b985da047a7aec85bc3aa4f1957'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.125842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77fc5b985da047a7aec85bc3aa4f1957'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.187867 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77fc5b985da047a7aec85bc3aa4f1957'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.194395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1262/2515] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:30:56 2022] DELETE /identity/v3/users/77fc5b985da047a7aec85bc3aa4f1957 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:30:57.453123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:57.454769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:57.468482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307320c-c4be-45a7-931b-8d7121dd9bcc admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:57.526618 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:57.527486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:57.527771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:57.528053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] PATH_INFO: `/v3/projects/2e343093ea7146318468d8c7e39f3703` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:57.536300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:57.538789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e343093ea7146318468d8c7e39f3703)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:57.540715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:57.666634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:57.668189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:57.671923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e343093ea7146318468d8c7e39f3703', 'request_id': 'req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.739156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62039b9b-e83c-4e19-8897-add57d95ab20 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:57.754722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e343093ea7146318468d8c7e39f3703', 'request_id': 'req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.761506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e343093ea7146318468d8c7e39f3703', 'request_id': 'req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.774791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] The token cache is being invalidate because project 2e343093ea7146318468d8c7e39f3703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:30:57.775079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.775772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-03da7a8b-5551-4f4e-bef9-7db3cddcaa5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:30:57.777938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1254/2516] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:30:57 2022] DELETE /identity/v3/projects/2e343093ea7146318468d8c7e39f3703 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:30:57.782749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237c5b0-e6d3-41ce-af53-a277a5c98107 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:57.783032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237c5b0-e6d3-41ce-af53-a277a5c98107 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:57.783351 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237c5b0-e6d3-41ce-af53-a277a5c98107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:57.792492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea4a306-bfa5-4221-90d3-cf673a359ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:57.798786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a306-bfa5-4221-90d3-cf673a359ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:57.798786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a306-bfa5-4221-90d3-cf673a359ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:57.798786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea4a306-bfa5-4221-90d3-cf673a359ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:57.798786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-0ea4a306-bfa5-4221-90d3-cf673a359ef2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:57.799175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:57.843383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d237c5b0-e6d3-41ce-af53-a277a5c98107 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:30:57.844757 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d237c5b0-e6d3-41ce-af53-a277a5c98107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:57.870786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4a306-bfa5-4221-90d3-cf673a359ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:57.872285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea4a306-bfa5-4221-90d3-cf673a359ef2 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:57.882124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1263/2517] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:30:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:30:57.906528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1255/2518] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:30:57.914382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d3ac-937b-4f1d-94b6-6187e7f39c6e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:57.914916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d3ac-937b-4f1d-94b6-6187e7f39c6e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:57.915377 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7d3ac-937b-4f1d-94b6-6187e7f39c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:57.934664 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-27a7d3ac-937b-4f1d-94b6-6187e7f39c6e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:30:57.935933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-27a7d3ac-937b-4f1d-94b6-6187e7f39c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:57.944367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1264/2519] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:30:57.948695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237c5b0-e6d3-41ce-af53-a277a5c98107 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:57.958270 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237c5b0-e6d3-41ce-af53-a277a5c98107 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:57.964072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:57.964705 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:57.965054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:57.965438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:57.966255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:30:57.966567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:57.968308 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:57.984410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1256/2520] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:30:57.985643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7d3ac-937b-4f1d-94b6-6187e7f39c6e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:57.995995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7d3ac-937b-4f1d-94b6-6187e7f39c6e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:58.000647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:58.001331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.001627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.001907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.002618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:58.004534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:58.044871 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:58.046580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:58.059835 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eaac81-48e1-4208-87ff-38a8b2eb7b29 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:58.063946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:58.064596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.064880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.065161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.066941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:58.106146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1265/2521] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:30:57 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:30:58.110610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:58.111290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:58.140311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:58.143553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:58.153148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1257/2522] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:30:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:30:58.183818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a61bcf6-7ab5-46c5-8e25-c5299b6065fd admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:58.188993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:58.189698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.189989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.190271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.191006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:58.192954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:58.319937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1266/2523] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:30:58 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:30:58.323676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:58.335932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00485ba-d44e-49bf-9ba8-193d8262913b service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:58.341380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:58.341951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.342428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.342566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.343215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:58.344790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:58.358450 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1258/2524] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:30:58.359787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:58.379154 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111ab1c-d2ca-4c45-9218-613d190b6572 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:58.384774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111e55b-0189-4495-982d-bc875d6f10d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:58.386088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111e55b-0189-4495-982d-bc875d6f10d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.386535 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111e55b-0189-4495-982d-bc875d6f10d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.386820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111e55b-0189-4495-982d-bc875d6f10d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.387560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111e55b-0189-4495-982d-bc875d6f10d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:58.389161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111e55b-0189-4495-982d-bc875d6f10d0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:58.405586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1267/2525] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:30:58.406956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:58.421169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99eb740-cf48-45ae-a2e3-0c566244b024 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:58.436917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:58.437936 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.443055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.443055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] PATH_INFO: `/v3/projects/31a105514cfa456d9de3a0b5903b2568/users/5a12e3c1ed3f41b699a5379d120396da/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.454296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a105514cfa456d9de3a0b5903b2568, user_id=5a12e3c1ed3f41b699a5379d120396da, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:58.456941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:58.630526 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1259/2526] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:30:58 2022] PUT /identity/v3/projects/31a105514cfa456d9de3a0b5903b2568/users/5a12e3c1ed3f41b699a5379d120396da/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:30:58.636266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.636698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.636979 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.655693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:30:58.656884 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:58.687338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:30:58.716779 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1268/2527] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:30:58.727430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:30:58.729363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:30:58.787057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ccbe6-fe2a-4a27-acb7-1ed59a2a8c7c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:30:58.790682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c3e6f-09e0-48c7-9964-781f99859a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:30:58.791361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c3e6f-09e0-48c7-9964-781f99859a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:30:58.791637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c3e6f-09e0-48c7-9964-781f99859a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:30:58.791911 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c3e6f-09e0-48c7-9964-781f99859a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:30:58.802324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c3e6f-09e0-48c7-9964-781f99859a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:30:58.803786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c3e6f-09e0-48c7-9964-781f99859a86 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:30:58.816620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1260/2528] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:30:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:31:00.676718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:00.678405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:00.800612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171c4b8-0ecb-48ef-9fa8-ad8aa4cf4f35 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:00.808818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e77f6-b44b-4f71-8978-4af81066bd2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:00.809550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e77f6-b44b-4f71-8978-4af81066bd2e service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:00.809896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e77f6-b44b-4f71-8978-4af81066bd2e service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:00.810183 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e77f6-b44b-4f71-8978-4af81066bd2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:00.812403 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7d2e77f6-b44b-4f71-8978-4af81066bd2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:00.832942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e77f6-b44b-4f71-8978-4af81066bd2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:00.835236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e77f6-b44b-4f71-8978-4af81066bd2e service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:00.855924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1269/2529] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:31:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:31:05.058631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d560-2334-47e5-8d7b-be2d69887b9c None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.062237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d560-2334-47e5-8d7b-be2d69887b9c None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.062686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d560-2334-47e5-8d7b-be2d69887b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.115959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.115959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.115959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.122603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-aa53d560-2334-47e5-8d7b-be2d69887b9c None None] MFA Rules not processed for user `5a12e3c1ed3f41b699a5379d120396da`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:31:05.124164 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-aa53d560-2334-47e5-8d7b-be2d69887b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:05.215144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:31:05.215144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:05.233843 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1261/2530] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:31:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:31:05.239245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53d560-2334-47e5-8d7b-be2d69887b9c None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:05.254361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53d560-2334-47e5-8d7b-be2d69887b9c None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:05.259177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:05.259854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.260392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.260564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.261155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:05.263136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:05.293862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1270/2531] 10.222.0.35 () {58 vars in 967 bytes} [Tue May 24 10:31:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:31:05.338011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:05.340029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:05.368930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:05.392534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1262/2532] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:31:05 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:31:05.396845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:05.407561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:05.457237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2a2ad-2ad7-48c7-9358-a8d1dae51ff7 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:05.469051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c7348e-05cf-4818-adcf-24188a9167ec admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:05.474215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:05.475157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.475609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.475996 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.477271 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:05.479482 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:05.526964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:05.527864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.528155 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.528437 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.605393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1263/2533] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:31:05 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:31:05.608942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:05.619053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:05.621511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:05.628128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72d3ca-6584-4212-87a0-38015f3ebe00 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:05.629828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1271/2534] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:31:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 10:31:05.633949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:05.634770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.635756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.636086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.637055 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:05.638891 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:05.651980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1264/2535] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:31:05.656238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:05.668479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638578c6-a961-4758-b53a-a68d23edf7a7 service swift] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:05.673723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:05.674733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.675082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.675430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] PATH_INFO: `/v3/projects/9f63e72ac1c74db6b014c616f523cfab/users/237774adfb8243babd59baa935e37170/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.693820 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f63e72ac1c74db6b014c616f523cfab, user_id=237774adfb8243babd59baa935e37170, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:05.696511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:05.814025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1272/2536] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:31:05 2022] PUT /identity/v3/projects/9f63e72ac1c74db6b014c616f523cfab/users/237774adfb8243babd59baa935e37170/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:31:05.818133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:05.829003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04649ae-cada-4c54-829b-d56706cac017 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:05.938128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:05.938821 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.939107 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.939388 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.940297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:05.941910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:05.957864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1265/2537] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:31:05.961980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:05.973989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbef192-85e8-47fe-8679-4bf1e5baf398 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:05.982165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:05.982165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:05.982165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:05.982165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] PATH_INFO: `/v3/projects/9f63e72ac1c74db6b014c616f523cfab/users/237774adfb8243babd59baa935e37170/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:05.983355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f63e72ac1c74db6b014c616f523cfab, user_id=237774adfb8243babd59baa935e37170, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:05.986083 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:05.997805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1273/2538] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:31:05 2022] PUT /identity/v3/projects/9f63e72ac1c74db6b014c616f523cfab/users/237774adfb8243babd59baa935e37170/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:31:06.725177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:06.726937 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:06.787007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdedb9e-efb1-4431-81dd-13e08a698dab admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:06.828779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:06.845305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34789e77-557f-48fa-8519-3b75541f1c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:06.845989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789e77-557f-48fa-8519-3b75541f1c3d service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:06.846274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789e77-557f-48fa-8519-3b75541f1c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:06.846554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34789e77-557f-48fa-8519-3b75541f1c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:06.849253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-34789e77-557f-48fa-8519-3b75541f1c3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:06.925489 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34789e77-557f-48fa-8519-3b75541f1c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:06.927032 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34789e77-557f-48fa-8519-3b75541f1c3d service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:06.938431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1266/2539] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:31:12.247101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:12.248831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:12.271669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:12.363100 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185c3ea-5976-4788-b041-14620f48aea9 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:12.368467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:12.369146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:12.369873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:12.370156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:12.372652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:12.387667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:12.389577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:12.405510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1274/2540] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:31:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:31:13.624383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:13.628954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:13.628954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:13.654018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] MFA Rules not processed for user `237774adfb8243babd59baa935e37170`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:31:13.655221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:13.737993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1267/2541] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:31:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:31:13.751856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:13.830487 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef6dc9-4fb6-4cbe-a274-b6fa819a6c03 service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:13.836132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:13.836796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:13.837081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:13.837389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:13.899328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:13.901843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:13.915401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1275/2542] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:31:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:31:16.917541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:16.919350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:16.939684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1131cf-bcfc-4b65-98b9-8d01a091aa4f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:16.944422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:16.947274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:16.947274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:16.947274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] PATH_INFO: `/v3/users/77f178a1600c4704a9f7dba7e3b8f36a` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:16.962710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=77f178a1600c4704a9f7dba7e3b8f36a)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:16.965976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:17.168187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77f178a1600c4704a9f7dba7e3b8f36a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.168903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77f178a1600c4704a9f7dba7e3b8f36a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.178134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77f178a1600c4704a9f7dba7e3b8f36a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.179267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77f178a1600c4704a9f7dba7e3b8f36a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.181179 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:17.196174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77f178a1600c4704a9f7dba7e3b8f36a'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.313627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1268/2543] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:31:16 2022] DELETE /identity/v3/users/77f178a1600c4704a9f7dba7e3b8f36a => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:31:17.591548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:17.604486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad906fe-09bd-4836-b929-2ee514cf2e36 service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:17.640668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:17.666072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:17.669364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:17.669651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:17.669923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] PATH_INFO: `/v3/projects/2606d8cef9c442c896c12b401c92326f` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:17.671144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2606d8cef9c442c896c12b401c92326f)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:17.673254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:17.805739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2606d8cef9c442c896c12b401c92326f', 'request_id': 'req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.943459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2606d8cef9c442c896c12b401c92326f', 'request_id': 'req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.950847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2606d8cef9c442c896c12b401c92326f', 'request_id': 'req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.961409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] The token cache is being invalidate because project 2606d8cef9c442c896c12b401c92326f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:31:17.961704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.962348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:17.964067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1276/2544] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:31:17 2022] DELETE /identity/v3/projects/2606d8cef9c442c896c12b401c92326f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:31:17.967994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:17.969479 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:17.982957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ed2d0-f2e1-4f72-a858-c6705647ad6a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:18.039054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:18.039930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:18.040210 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:18.040485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] PATH_INFO: `/v3/users/58832bf1d66342d7bad90d67f9272e4f` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:18.053429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58832bf1d66342d7bad90d67f9272e4f)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:18.055340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:18.351423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58832bf1d66342d7bad90d67f9272e4f'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:18.352093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58832bf1d66342d7bad90d67f9272e4f'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:18.359764 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58832bf1d66342d7bad90d67f9272e4f'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:18.360444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58832bf1d66342d7bad90d67f9272e4f'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:18.374600 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58832bf1d66342d7bad90d67f9272e4f'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:18.463281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1269/2545] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:31:17 2022] DELETE /identity/v3/users/58832bf1d66342d7bad90d67f9272e4f => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:31:18.727503 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:18.729010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:18.746856 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b32d4d7-ee89-4cd7-8f91-472599f38fa1 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:18.803347 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:18.804210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:18.804493 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:18.804765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] PATH_INFO: `/v3/projects/55c5945038624ca4ab268e5587bc002d` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:18.816614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c5945038624ca4ab268e5587bc002d)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:18.818607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:18.932667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c5945038624ca4ab268e5587bc002d', 'request_id': 'req-74af9a45-d741-4a4a-b05c-1e742b1ba286'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:19.017294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c5945038624ca4ab268e5587bc002d', 'request_id': 'req-74af9a45-d741-4a4a-b05c-1e742b1ba286'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:19.025678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c5945038624ca4ab268e5587bc002d', 'request_id': 'req-74af9a45-d741-4a4a-b05c-1e742b1ba286'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:19.027030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:19.036523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] The token cache is being invalidate because project 55c5945038624ca4ab268e5587bc002d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:31:19.036817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:19.037627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:31:19.039088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1277/2546] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:31:18 2022] DELETE /identity/v3/projects/55c5945038624ca4ab268e5587bc002d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:31:28.459499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:29.240214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=2,keystone:SELECT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:31:47.031883 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:31:47.033306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:47.101860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f8638-f239-4851-9624-2aa4bae79ee3 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:31:47.161694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:31:47.162366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:31:47.162682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:31:47.162965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:31:47.165137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:31:47.178269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:31:47.179403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:31:47.190545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1270/2547] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:31:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:31:57.185872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=58 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:24.911828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:32:24.913526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:32:24.977992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af9a45-d741-4a4a-b05c-1e742b1ba286 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:32:24.982466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:32:24.983335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:32:24.983614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:32:24.983895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] PATH_INFO: `/v3/users/5a12e3c1ed3f41b699a5379d120396da` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:32:24.985061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a12e3c1ed3f41b699a5379d120396da)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:32:24.987016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:32:25.152363 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a12e3c1ed3f41b699a5379d120396da'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.159442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a12e3c1ed3f41b699a5379d120396da'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.160555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a12e3c1ed3f41b699a5379d120396da'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.176667 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a12e3c1ed3f41b699a5379d120396da'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.262810 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a12e3c1ed3f41b699a5379d120396da'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.269507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1278/2548] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:32:24 2022] DELETE /identity/v3/users/5a12e3c1ed3f41b699a5379d120396da => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:32:25.544436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:32:25.556359 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52231fdc-d4d2-4415-9a28-d3bfb7c11e4b service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:32:25.620101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:32:25.620972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:32:25.621293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:32:25.621575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] PATH_INFO: `/v3/projects/31a105514cfa456d9de3a0b5903b2568` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:32:25.622748 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a105514cfa456d9de3a0b5903b2568)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:32:25.624647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:32:25.777241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a105514cfa456d9de3a0b5903b2568', 'request_id': 'req-040fc799-c5aa-4108-8d19-d2204f0c1cf8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.862320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a105514cfa456d9de3a0b5903b2568', 'request_id': 'req-040fc799-c5aa-4108-8d19-d2204f0c1cf8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.868183 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a105514cfa456d9de3a0b5903b2568', 'request_id': 'req-040fc799-c5aa-4108-8d19-d2204f0c1cf8'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.879864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] The token cache is being invalidate because project 31a105514cfa456d9de3a0b5903b2568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:32:25.880153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.880864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:25.882416 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1271/2549] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:32:25 2022] DELETE /identity/v3/projects/31a105514cfa456d9de3a0b5903b2568 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:32:25.886552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:32:25.887923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:32:25.901408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e92e8d-afed-4c5d-a084-cb213dcf5369 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:32:25.959751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:32:25.960642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:32:25.960928 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:32:25.961205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] PATH_INFO: `/v3/users/237774adfb8243babd59baa935e37170` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:32:25.962428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=237774adfb8243babd59baa935e37170)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:32:25.964348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:32:25.985520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:26.184769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237774adfb8243babd59baa935e37170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.194112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237774adfb8243babd59baa935e37170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.194788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237774adfb8243babd59baa935e37170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.206556 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237774adfb8243babd59baa935e37170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.302569 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237774adfb8243babd59baa935e37170'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.308805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1279/2550] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:32:25 2022] DELETE /identity/v3/users/237774adfb8243babd59baa935e37170 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:32:26.556485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:32:26.557973 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:32:26.572321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040fc799-c5aa-4108-8d19-d2204f0c1cf8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:32:26.632158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:32:26.632992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:32:26.633277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:32:26.633577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] PATH_INFO: `/v3/projects/9f63e72ac1c74db6b014c616f523cfab` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:32:26.634749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f63e72ac1c74db6b014c616f523cfab)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:32:26.636645 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:32:26.761064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:26.773786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f63e72ac1c74db6b014c616f523cfab', 'request_id': 'req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.845589 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f63e72ac1c74db6b014c616f523cfab', 'request_id': 'req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.852072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f63e72ac1c74db6b014c616f523cfab', 'request_id': 'req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.863122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] The token cache is being invalidate because project 9f63e72ac1c74db6b014c616f523cfab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:32:26.863409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.864057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:32:26.865622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1272/2551] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:32:26 2022] DELETE /identity/v3/projects/9f63e72ac1c74db6b014c616f523cfab => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:32:36.305057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:32:36.977514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:33:56.755505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:33:56.757625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:33:56.830136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f66901-fafe-4b54-8863-8c1ee77d7ebe admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:33:56.891322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:33:56.891973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:33:56.892259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:33:56.892541 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:33:56.894324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:33:56.961003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:33:56.962635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:33:56.980828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1280/2552] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:33:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:33:58.126004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:33:58.127403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:33:58.185180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b834135-7d6f-4c78-8e5c-fc0face8b6aa admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:33:58.188801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:33:58.189478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:33:58.189763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:33:58.190041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:33:58.192048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:33:58.206101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:33:58.207641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:33:58.218957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1273/2553] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:33:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:34:06.978228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:08.214077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=26 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:34.410134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:34:34.411620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:34:34.423641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd9346-444a-4fc0-9a43-10dd6a6cae06 service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:34:34.427174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:34:34.427816 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:34:34.428101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:34:34.428383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:34:34.430824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:34:34.444094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:34:34.445691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:34:34.456557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1281/2554] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:34:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:34:44.451697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:34:58.509395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:34:58.511463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:34:58.566274 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b0a7e-76bf-46fa-8072-a3bdcf3f951b service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:34:58.570220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:34:58.570864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:34:58.571145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:34:58.571419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:34:58.573736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:34:58.633431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:34:58.635824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:34:58.644824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1274/2555] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:34:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:35:08.640661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=48 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:45.865033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:35:45.874295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef6643-663a-4c6c-8db5-e02925c19728 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:35:45.877931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:35:45.878640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:35:45.878922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:35:45.879203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:35:45.881016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:35:45.891564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:35:45.892991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:35:45.901137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1282/2556] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:35:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:35:55.897544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:35:57.955365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:35:57.974377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b080b2-32b1-4c16-a94c-5355387df9fc service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:35:57.978013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:35:57.978650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:35:57.978933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:35:57.979207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:35:57.981075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:35:58.062637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:35:58.064319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:35:58.083141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1275/2557] 10.222.0.35 () {60 vars in 1396 bytes} [Tue May 24 10:35:57 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:36:08.080563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=47 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:36:49.182409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:36:49.183798 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:36:49.287307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587b0b87-a56e-451c-ae83-ad54e60e21a9 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:36:49.292297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:36:49.292967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:36:49.293249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:36:49.293560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:36:49.295317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:36:49.306982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:36:49.308545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:36:49.318818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1283/2558] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:36:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:36:59.314050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=48 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:02.453918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:02.455293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:02.467494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2527293-f559-4cc9-9921-991b9154b5c5 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:02.471024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:02.471661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:02.471943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:02.472218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:02.473989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:02.541978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:02.543408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:02.561837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1276/2559] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:37:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:37:02.672681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:02.681785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585eb06e-8006-47fa-ad77-71626d14b867 service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:02.685298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ddecf-2aa4-4abd-ae28-1898d8fa2f41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:02.685929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ddecf-2aa4-4abd-ae28-1898d8fa2f41 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:02.686206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ddecf-2aa4-4abd-ae28-1898d8fa2f41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:02.686478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ddecf-2aa4-4abd-ae28-1898d8fa2f41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:02.688210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f32ddecf-2aa4-4abd-ae28-1898d8fa2f41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application [None req-f32ddecf-2aa4-4abd-ae28-1898d8fa2f41 service cinder] Could not find project: 5bf4c198860648cd874233ba27d1b971.: keystone.exception.ProjectNotFound: Could not find project: 5bf4c198860648cd874233ba27d1b971. May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 24 10:37:02.714102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 10:37:02.715411 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self._validate_token_resources() May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 10:37:02.716702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.get_or_create( May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application with Lock( May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._enter() May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.creator() May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application created_value = creator( May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 10:37:02.718007 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 10:37:02.719317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5bf4c198860648cd874233ba27d1b971. May 24 10:37:02.719317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: ERROR keystone.server.flask.application May 24 10:37:02.719317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1284/2560] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:37:02 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 24 10:37:02.806832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:02.808528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:02.878906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eb704-8618-46e2-b653-3e7f9ebf0108 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:02.884562 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:02.885456 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:02.885752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:02.886007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] PATH_INFO: `/v3/users/e8f1082f78d64bb69ed4c8e5cd2e75dc` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:02.887191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8f1082f78d64bb69ed4c8e5cd2e75dc)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:02.889082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:03.023269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8f1082f78d64bb69ed4c8e5cd2e75dc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.023848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8f1082f78d64bb69ed4c8e5cd2e75dc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.030773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8f1082f78d64bb69ed4c8e5cd2e75dc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.032346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8f1082f78d64bb69ed4c8e5cd2e75dc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.043487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8f1082f78d64bb69ed4c8e5cd2e75dc'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.125009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:03.129322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1277/2561] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:37:02 2022] DELETE /identity/v3/users/e8f1082f78d64bb69ed4c8e5cd2e75dc => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:37:03.134435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:03.135522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:03.135522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:03.156209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:37:03.157438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:03.218885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1285/2562] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:37:03.228742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:03.230206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:03.288682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58be7e76-7c44-4299-825e-e42232be8520 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:03.292224 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e41161-b363-44e9-bc6b-64a1ca33c7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:03.292866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41161-b363-44e9-bc6b-64a1ca33c7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:03.293152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41161-b363-44e9-bc6b-64a1ca33c7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:03.293459 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e41161-b363-44e9-bc6b-64a1ca33c7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:03.302228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e41161-b363-44e9-bc6b-64a1ca33c7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:03.304462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e41161-b363-44e9-bc6b-64a1ca33c7a9 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:03.313683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1278/2563] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:37:03.750551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:03.754151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:03.764868 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68556f-b555-4905-b949-1c2b834dd663 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:03.769458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:03.770408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:03.770737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:03.771012 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] PATH_INFO: `/v3/projects/28478e7f51704defa09c87ffc818cd18` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:03.772821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] RBAC: Authorizing `identity:delete_project(project_id=28478e7f51704defa09c87ffc818cd18)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:03.774785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:03.871954 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28478e7f51704defa09c87ffc818cd18', 'request_id': 'req-4d7e605d-309b-4462-bb41-3e43c943370c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.926436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28478e7f51704defa09c87ffc818cd18', 'request_id': 'req-4d7e605d-309b-4462-bb41-3e43c943370c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.933743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28478e7f51704defa09c87ffc818cd18', 'request_id': 'req-4d7e605d-309b-4462-bb41-3e43c943370c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.945709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] The token cache is being invalidate because project 28478e7f51704defa09c87ffc818cd18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:37:03.945709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.945943 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-4d7e605d-309b-4462-bb41-3e43c943370c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:37:03.947574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1286/2564] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:37:03 2022] DELETE /identity/v3/projects/28478e7f51704defa09c87ffc818cd18 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:37:03.953383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d73f-d480-4b19-b1a8-e7664b69d1f8 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:03.953874 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d73f-d480-4b19-b1a8-e7664b69d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:03.953983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1d73f-d480-4b19-b1a8-e7664b69d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:03.971699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b2e1d73f-d480-4b19-b1a8-e7664b69d1f8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:37:03.972906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b2e1d73f-d480-4b19-b1a8-e7664b69d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:04.033153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1279/2565] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:37:04.041209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137b1f3-2900-4211-8544-9e2b372613b2 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.041522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137b1f3-2900-4211-8544-9e2b372613b2 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.041861 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137b1f3-2900-4211-8544-9e2b372613b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.060825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2137b1f3-2900-4211-8544-9e2b372613b2 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:37:04.062044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2137b1f3-2900-4211-8544-9e2b372613b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:04.070643 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1287/2566] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:37:04.075026 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d73f-d480-4b19-b1a8-e7664b69d1f8 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:04.084694 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1d73f-d480-4b19-b1a8-e7664b69d1f8 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:04.089290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:04.089956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.090237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.090516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.091245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:37:04.091552 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:04.093159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:04.104421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1280/2567] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:37:04.108486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137b1f3-2900-4211-8544-9e2b372613b2 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:04.118218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137b1f3-2900-4211-8544-9e2b372613b2 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:04.122807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:04.123448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.123728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.123998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.124731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:04.126642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:04.213972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1288/2568] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:37:04.218159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:04.228297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106152a-4921-4c14-8451-19741b9d37b8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:04.285394 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:04.286059 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.286341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.286619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.287304 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:04.289402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:04.370071 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1281/2569] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:37:04 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:37:04.374049 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:04.384421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189c549-4106-4b4e-94c5-fae1e75e6587 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:04.389051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:04.389740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.390021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.390293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.391170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:04.392761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:04.404676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1289/2570] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:37:04.408533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:04.418930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7fa07-b695-47f8-acdf-18868733245f admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:04.423506 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99310bca-1e24-4ddb-a7f8-efc0d1d072ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:04.424152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99310bca-1e24-4ddb-a7f8-efc0d1d072ff admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.424437 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99310bca-1e24-4ddb-a7f8-efc0d1d072ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.424714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99310bca-1e24-4ddb-a7f8-efc0d1d072ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.425485 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99310bca-1e24-4ddb-a7f8-efc0d1d072ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:04.427069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99310bca-1e24-4ddb-a7f8-efc0d1d072ff admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:04.437511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1282/2571] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:37:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:37:04.441718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:37:04.451759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd677bd0-cefd-4360-b1d5-78d73a941736 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:37:04.456299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:37:04.457160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.457468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.457741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] PATH_INFO: `/v3/projects/dac302ca572f48cf9c4b2a9d3185d4c4/users/1104ab5fbd5040a98f47d08a042dbd70/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.471804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dac302ca572f48cf9c4b2a9d3185d4c4, user_id=1104ab5fbd5040a98f47d08a042dbd70, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:37:04.474460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:37:04.591290 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1290/2572] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:37:04 2022] PUT /identity/v3/projects/dac302ca572f48cf9c4b2a9d3185d4c4/users/1104ab5fbd5040a98f47d08a042dbd70/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:37:04.601530 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bb9ac-cf9d-46e6-b1cb-5ea20938d6ab None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:37:04.601824 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bb9ac-cf9d-46e6-b1cb-5ea20938d6ab None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:37:04.602134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bb9ac-cf9d-46e6-b1cb-5ea20938d6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:37:04.621209 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:04.633257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d09bb9ac-cf9d-46e6-b1cb-5ea20938d6ab None None] MFA Rules not processed for user `1104ab5fbd5040a98f47d08a042dbd70`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:37:04.633257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d09bb9ac-cf9d-46e6-b1cb-5ea20938d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:37:04.699133 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1283/2573] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:37:14.587015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:37:14.695838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=28 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:04.765899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:38:04.825119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebfb07a-45e2-49f7-85af-793fec0e8a92 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:38:04.830342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:38:04.831210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:38:04.831491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:38:04.831765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] PATH_INFO: `/v3/users/1104ab5fbd5040a98f47d08a042dbd70` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:38:04.832929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1104ab5fbd5040a98f47d08a042dbd70)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:38:04.834870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:38:04.970594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1104ab5fbd5040a98f47d08a042dbd70'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:04.977356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1104ab5fbd5040a98f47d08a042dbd70'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:04.978014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1104ab5fbd5040a98f47d08a042dbd70'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:04.990567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1104ab5fbd5040a98f47d08a042dbd70'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:05.052343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1104ab5fbd5040a98f47d08a042dbd70'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:05.058348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1291/2574] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:38:04 2022] DELETE /identity/v3/users/1104ab5fbd5040a98f47d08a042dbd70 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:38:05.063636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:38:05.064232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:38:05.064232 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:38:05.083780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:38:05.085051 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:05.152067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1284/2575] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:38:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:38:05.163291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:38:05.164781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:05.226741 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b01658-8010-4b81-92f4-89889317cb32 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:38:05.230515 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:38:05.234098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:38:05.234098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:38:05.234098 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:38:05.241674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:38:05.243243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:38:05.254042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1292/2576] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:38:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:38:05.662277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:38:05.663691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:05.676182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03195a26-189b-46ca-bd02-9cb9ed9b834a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:38:05.681699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:38:05.682742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:38:05.683022 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:38:05.683294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] PATH_INFO: `/v3/projects/dac302ca572f48cf9c4b2a9d3185d4c4` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:38:05.684504 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dac302ca572f48cf9c4b2a9d3185d4c4)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:38:05.688579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:38:05.784131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dac302ca572f48cf9c4b2a9d3185d4c4', 'request_id': 'req-f7390a0b-8ca2-4080-9363-0b56f8c84737'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:05.840864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dac302ca572f48cf9c4b2a9d3185d4c4', 'request_id': 'req-f7390a0b-8ca2-4080-9363-0b56f8c84737'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:05.846634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dac302ca572f48cf9c4b2a9d3185d4c4', 'request_id': 'req-f7390a0b-8ca2-4080-9363-0b56f8c84737'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:05.858040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] The token cache is being invalidate because project dac302ca572f48cf9c4b2a9d3185d4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:38:05.858329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:05.858957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:38:05.860498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1285/2577] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:38:05 2022] DELETE /identity/v3/projects/dac302ca572f48cf9c4b2a9d3185d4c4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:38:15.250021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:15.854472 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:57.756732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:38:57.758138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:57.779180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26419a5c-388b-4823-bfd9-4f46bd61a8a4 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:38:57.825220 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:38:57.825886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:38:57.826165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:38:57.826436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:38:57.828156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:57.915808 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:38:57.917253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:38:57.937460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1293/2578] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:38:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:38:58.827385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:38:58.828794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:58.896134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7390a0b-8ca2-4080-9363-0b56f8c84737 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:38:58.899767 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:38:58.900410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:38:58.900696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:38:58.900977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:38:58.902751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] Could not find project: dac302ca572f48cf9c4b2a9d3185d4c4.: keystone.exception.ProjectNotFound: Could not find project: dac302ca572f48cf9c4b2a9d3185d4c4. May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 24 10:38:58.915882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 10:38:58.917143 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._validate_token_resources() May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 10:38:58.918476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 10:38:58.919514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 10:38:58.920579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: dac302ca572f48cf9c4b2a9d3185d4c4. May 24 10:38:58.920579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 10:38:58.920579 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1286/2579] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:38:58 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 90 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 24 10:38:58.928706 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:38:58.938338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8930f7a-514a-4d62-8183-9e9cdae9ae7a service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:38:58.942519 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:38:58.943174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:38:58.943462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:38:58.943757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:38:58.945576 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:38:59.015496 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:38:59.019923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:38:59.021484 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:38:59.030489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1294/2580] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:38:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:39:08.906379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=28 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:39:09.054624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:40:58.354023 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:40:58.355520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:40:58.373234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219eeb41-d56a-4d0c-929b-cdbf6a7914af service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:40:58.378664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:40:58.379301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:40:58.379587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:40:58.379867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:40:58.381669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:40:58.453796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:40:58.455440 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:40:58.464564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1287/2581] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:40:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:41:00.410130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:41:00.417562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12673b-8514-4e0c-ab60-3b2cb2dd32be service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:41:00.423592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:41:00.424324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:41:00.424601 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:41:00.424875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:41:00.426653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:41:00.439394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:41:00.441061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:41:00.452237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1295/2582] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:41:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:41:08.460369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=33 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:41:10.447016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:41:51.853114 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:41:51.854695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:41:51.888191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e22f-86a9-4b62-bffd-ac3b235ac81f service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:41:51.891766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:41:51.892405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:41:51.892685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:41:51.892959 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:41:51.894904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:41:51.904123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:41:51.905555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:41:51.914206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1288/2583] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:41:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:41:55.177344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:41:55.178766 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:41:55.259182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff149172-0011-4d3d-9ecb-287860b60db7 service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:41:55.264082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:41:55.264729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:41:55.265013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:41:55.265316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:41:55.267075 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:41:55.278558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:41:55.280170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:41:55.289995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1296/2584] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:41:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:42:01.910775 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=15 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:42:05.285870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=37 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:00.938623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:00.948263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53780808-9712-4d17-b49c-abb952243397 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:00.951975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:00.952622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:00.952904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:00.953184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:00.955061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:01.028604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:01.030091 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:01.040715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1289/2585] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:43:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:43:03.270131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:03.271567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:03.283926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2e4636-4418-4682-8162-20d3dc570632 service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:03.288833 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:03.289507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:03.289786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:03.290059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:03.291896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:03.379172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:03.380540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:03.400086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1297/2586] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:43:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:43:06.855918 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:06.857453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:06.872993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73cda44-0983-4472-a1e8-352e23f0f1ba service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:06.878106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:06.878760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:06.879048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:06.880162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:06.882334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:06.894205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:06.895919 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:06.906449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1290/2587] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:43:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:43:06.981476 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:06.988191 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:06.996197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ced342-5a88-4874-a4d9-fac847d74410 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:07.001020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:07.001933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:07.002222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:07.002505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] PATH_INFO: `/v3/users/7bffd469507b4dbc8d3146f051c8c8ad` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:07.003700 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bffd469507b4dbc8d3146f051c8c8ad)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:07.005679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:07.190448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bffd469507b4dbc8d3146f051c8c8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.199665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bffd469507b4dbc8d3146f051c8c8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.200373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bffd469507b4dbc8d3146f051c8c8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.217799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bffd469507b4dbc8d3146f051c8c8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.284707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c42480f3-899e-4331-864f-ebdc1dc9ad9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bffd469507b4dbc8d3146f051c8c8ad'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.291119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1298/2588] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:43:06 2022] DELETE /identity/v3/users/7bffd469507b4dbc8d3146f051c8c8ad => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:43:07.520276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:07.532571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b3409-cd32-41e4-9520-0a4878c8470d service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:07.592478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:07.593366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:07.593639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:07.593914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] PATH_INFO: `/v3/projects/a02e2e12ebd14e5d8e960a65f41bfb00` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:07.595138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02e2e12ebd14e5d8e960a65f41bfb00)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:07.597057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:07.712914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'request_id': 'req-e794b1bf-8d4d-4ef4-875d-517ca0673946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.786159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'request_id': 'req-e794b1bf-8d4d-4ef4-875d-517ca0673946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.792014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02e2e12ebd14e5d8e960a65f41bfb00', 'request_id': 'req-e794b1bf-8d4d-4ef4-875d-517ca0673946'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.802979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] The token cache is being invalidate because project a02e2e12ebd14e5d8e960a65f41bfb00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:43:07.803268 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.805080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e794b1bf-8d4d-4ef4-875d-517ca0673946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:43:07.806675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1291/2589] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:43:07 2022] DELETE /identity/v3/projects/a02e2e12ebd14e5d8e960a65f41bfb00 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:43:07.812330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13993c23-9c88-4be8-b1af-19d48287799d None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:07.812718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13993c23-9c88-4be8-b1af-19d48287799d None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:07.812891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13993c23-9c88-4be8-b1af-19d48287799d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:07.864694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:07.868295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-13993c23-9c88-4be8-b1af-19d48287799d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:43:07.869548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-13993c23-9c88-4be8-b1af-19d48287799d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:07.926712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1299/2590] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:43:07.934710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91125-1be6-4251-8e03-6a6819be9295 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:07.935198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91125-1be6-4251-8e03-6a6819be9295 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:07.935324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a91125-1be6-4251-8e03-6a6819be9295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:07.942445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:07.953182 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-26a91125-1be6-4251-8e03-6a6819be9295 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:43:07.954405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-26a91125-1be6-4251-8e03-6a6819be9295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:07.962984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1292/2591] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:43:07.967112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13993c23-9c88-4be8-b1af-19d48287799d None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:07.977403 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13993c23-9c88-4be8-b1af-19d48287799d None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:07.982804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:07.990428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:07.990428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:07.990428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:07.990428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:43:07.990428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:07.990428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:07.998780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1300/2592] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:43:08.003019 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a91125-1be6-4251-8e03-6a6819be9295 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.012405 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a91125-1be6-4251-8e03-6a6819be9295 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.017103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.019253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.019253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.019253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.019253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.020863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.124865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1293/2593] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:43:08 2022] POST /identity/v3/projects => generated 357 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:43:08.129125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.139792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f0be4-112d-403f-b838-dd306b832abb admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.196918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.197635 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.197922 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.198204 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.198924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.200858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.289314 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1301/2594] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:43:08 2022] POST /identity/v3/users => generated 279 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:43:08.293542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.303770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef6cef-b51c-4e9d-972d-0256b2e13fd3 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.308668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.309361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.309648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.309927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.310661 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.312246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.322814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1294/2595] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:43:08.326650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.336815 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48505f69-657a-43ef-96b7-3760688557da admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.342193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31155913-a20c-44ff-8a99-e780d82482fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.342852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31155913-a20c-44ff-8a99-e780d82482fa admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.343136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31155913-a20c-44ff-8a99-e780d82482fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.343412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31155913-a20c-44ff-8a99-e780d82482fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.344141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31155913-a20c-44ff-8a99-e780d82482fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.345786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31155913-a20c-44ff-8a99-e780d82482fa admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.355743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1302/2596] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:43:08.359755 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.369626 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd11569-bdc7-4307-93f4-2b884f404b7c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.374216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.375088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.375373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.375649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] PATH_INFO: `/v3/projects/24859ed8e2ae4dc4bb93b8fc62d5aece/users/724673301f6549bdb23726232a962daf/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.390833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24859ed8e2ae4dc4bb93b8fc62d5aece, user_id=724673301f6549bdb23726232a962daf, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.393493 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.455313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1295/2597] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:43:08 2022] PUT /identity/v3/projects/24859ed8e2ae4dc4bb93b8fc62d5aece/users/724673301f6549bdb23726232a962daf/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:43:08.470940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5213-75a9-475a-8668-460eaf862fae None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.471233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5213-75a9-475a-8668-460eaf862fae None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.471509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce5213-75a9-475a-8668-460eaf862fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.499537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-18ce5213-75a9-475a-8668-460eaf862fae None None] MFA Rules not processed for user `724673301f6549bdb23726232a962daf`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:43:08.500666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-18ce5213-75a9-475a-8668-460eaf862fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:08.563604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1303/2598] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:43:08 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:43:08.568208 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.629354 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d36a63-bc82-474b-8b6c-11cd1bc355a9 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.634027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.634680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.634960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.635236 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.635929 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.637850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.783403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1296/2599] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:43:08 2022] POST /identity/v3/projects => generated 355 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:43:08.787256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5213-75a9-475a-8668-460eaf862fae None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.793693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:08.799274 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce5213-75a9-475a-8668-460eaf862fae None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.854491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.855156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.855438 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.855714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.856398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.858368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.952504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1304/2600] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:43:08 2022] POST /identity/v3/users => generated 284 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:43:08.956434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:08.968000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b66986-d4f9-41ff-9951-3bbb86df7eb8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:08.972906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:08.973687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:08.973943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:08.974223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:08.974956 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:08.976581 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:08.987095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1297/2601] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:43:08.991273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:09.001355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90033f-52bb-4cf6-9f1f-a05740bd8c0d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:09.005933 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:09.006809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:09.007731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:09.007995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] PATH_INFO: `/v3/projects/a05957387bb9499d83736eb09ab03c95/users/e68b0ad5ee124a5e9ebadd513351238c/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:09.023413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05957387bb9499d83736eb09ab03c95, user_id=e68b0ad5ee124a5e9ebadd513351238c, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:09.026244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:09.089821 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1305/2602] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:43:08 2022] PUT /identity/v3/projects/a05957387bb9499d83736eb09ab03c95/users/e68b0ad5ee124a5e9ebadd513351238c/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:43:09.093784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:09.103873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e9b7e-8943-4090-8c0e-f150137c5f33 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:09.159370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2d62b-0b08-4962-9223-d99aa46699e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:09.160041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2d62b-0b08-4962-9223-d99aa46699e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:09.160324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2d62b-0b08-4962-9223-d99aa46699e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:09.160598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2d62b-0b08-4962-9223-d99aa46699e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:09.161349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2d62b-0b08-4962-9223-d99aa46699e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:09.163007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2d62b-0b08-4962-9223-d99aa46699e8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:09.173603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1298/2603] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:43:09.177756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:09.188400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda74f80-5acd-43da-ba9c-f70df9637f2e admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:09.193245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:09.194200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:09.195174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:09.195554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] PATH_INFO: `/v3/projects/a05957387bb9499d83736eb09ab03c95/users/e68b0ad5ee124a5e9ebadd513351238c/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:09.197702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a05957387bb9499d83736eb09ab03c95, user_id=e68b0ad5ee124a5e9ebadd513351238c, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:09.200394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:09.211963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1306/2604] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:43:09 2022] PUT /identity/v3/projects/a05957387bb9499d83736eb09ab03c95/users/e68b0ad5ee124a5e9ebadd513351238c/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:43:09.222089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72269-36ff-460e-9604-64cd59a23086 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:09.222384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72269-36ff-460e-9604-64cd59a23086 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:09.222697 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72269-36ff-460e-9604-64cd59a23086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:09.228796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:09.250597 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-04c72269-36ff-460e-9604-64cd59a23086 None None] MFA Rules not processed for user `e68b0ad5ee124a5e9ebadd513351238c`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:43:09.251810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-04c72269-36ff-460e-9604-64cd59a23086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:09.323582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1299/2605] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:43:09 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:43:09.334858 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:09.336990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:09.407240 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68325d4a-698a-490d-8cdb-bcce71b2f638 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:09.410897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:09.411544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:09.411822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:09.412097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:09.421244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:09.422684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:09.432364 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1307/2606] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:43:09 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:43:09.633627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c72269-36ff-460e-9604-64cd59a23086 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:09.643189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c72269-36ff-460e-9604-64cd59a23086 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:09.646886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:09.647522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:09.647861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:09.648172 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:09.704932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:09.706422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:09.715725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1300/2607] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:43:09 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:43:19.427777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:19.711648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=61 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:43:59.400712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:43:59.400712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d57b3-a875-40c6-8196-7a1d02788e66 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:43:59.403958 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:43:59.404592 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:43:59.404991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:43:59.405203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:43:59.406994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:43:59.474752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:43:59.476221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:43:59.495125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1308/2608] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:43:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:44:09.492316 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=36 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:45:58.617752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:45:58.619206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:45:58.631004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad270e-df5b-479f-80e6-14541eee3e27 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:45:58.634545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:45:58.635176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:45:58.635460 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:45:58.635735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:45:58.637454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:45:58.719056 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:45:58.721411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:45:58.742486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1301/2609] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:45:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:46:08.740306 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=42 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:46:57.864876 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:46:57.866440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:46:57.939378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35b773-0339-443b-b0cb-1892b8e5f6e9 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:46:57.942995 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf25fa8-68a7-4e20-8f4a-858ae7a3b89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:46:57.944213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf25fa8-68a7-4e20-8f4a-858ae7a3b89c service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:46:57.944213 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf25fa8-68a7-4e20-8f4a-858ae7a3b89c service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:46:57.944472 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf25fa8-68a7-4e20-8f4a-858ae7a3b89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:46:57.946165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5cf25fa8-68a7-4e20-8f4a-858ae7a3b89c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:46:57.955462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf25fa8-68a7-4e20-8f4a-858ae7a3b89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:46:57.956891 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf25fa8-68a7-4e20-8f4a-858ae7a3b89c service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:46:57.965697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1309/2610] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:46:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:46:58.918890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:46:58.920277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:46:59.013897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe18c71-33e7-45fb-a78e-e13399ecf974 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:46:59.018783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:46:59.020110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:46:59.020110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:46:59.020110 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:46:59.021813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:46:59.032874 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:46:59.034483 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:46:59.044912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1302/2611] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:47:07.961966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=35 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:47:09.040791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=44 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:03.637250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be9f0c-2cc9-4315-9541-a39ba641547c None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:03.637854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be9f0c-2cc9-4315-9541-a39ba641547c None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:03.637854 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be9f0c-2cc9-4315-9541-a39ba641547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:03.664231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a4be9f0c-2cc9-4315-9541-a39ba641547c None None] MFA Rules not processed for user `bb2d87f281834e4697a8c3559b6e83a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:03.665461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a4be9f0c-2cc9-4315-9541-a39ba641547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:03.672291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1310/2612] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:03.712130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:03.713850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:03.797457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198f518-138b-4d5e-b2bd-3fc6aa3b19bd service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:03.797457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:03.798004 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:03.798294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:03.798574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:03.800350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:03.870495 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:03.871939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:03.887469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1303/2613] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:48:04.089931 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be9f0c-2cc9-4315-9541-a39ba641547c None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:04.100651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be9f0c-2cc9-4315-9541-a39ba641547c None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:04.104410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:04.105904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:04.105904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:04.105904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:04.107904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:04.108167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:04.119457 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a54bf-2c28-404a-b027-9bc1769ce363 service glance] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:04.119696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:04.120993 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:04.123070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:04.123736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:04.124016 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:04.124292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:04.126097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:04.130029 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1311/2614] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] Could not find project: 31a105514cfa456d9de3a0b5903b2568.: keystone.exception.ProjectNotFound: Could not find project: 31a105514cfa456d9de3a0b5903b2568. May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application Traceback (most recent call last): May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 24 10:48:04.139144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 10:48:04.140372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self._validate_token_resources() May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 24 10:48:04.141623 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.get_or_create( May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application with Lock( May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._enter() May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.creator() May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application created_value = creator( May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 24 10:48:04.143543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 24 10:48:04.144928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 31a105514cfa456d9de3a0b5903b2568. May 24 10:48:04.144928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: ERROR keystone.server.flask.application May 24 10:48:04.144928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1304/2615] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:04 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 33 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 24 10:48:04.540165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:04.542327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:04.561771 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26933c0d-6302-4445-bc64-aa9c49a09a85 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:04.567811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:04.568589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:04.568894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:04.569173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:04.571146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:04.589785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:04.591489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:04.602916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1312/2616] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:06.140833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:06.140833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:06.156469 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac7bb9-87d1-4807-988c-4107cfded7d0 service glance] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:06.166965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:06.167640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:06.168064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:06.168348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:06.171049 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:06.185030 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:06.186770 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:06.198650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1305/2617] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:10.133789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:10.143332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a4ba4d-59d8-41f1-a5ce-04bf28539f07 service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:10.147136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:10.151734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:10.151734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:10.151734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:10.157144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:10.158571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:10.172916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1313/2618] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:10 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:48:11.612890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:11.622561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3f02f-64a8-43ef-82b5-bf5308113c60 service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:11.626596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:11.627228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:11.627509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:11.627787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:11.630269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:11.689542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:11.694332 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:11.695779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:11.706689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1306/2619] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:13.173822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:13.187394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d10aa-5870-4d5f-a685-0c7250e970f6 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:13.193663 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:13.194413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:13.194763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:13.195107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:13.197031 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:13.211474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:13.213385 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:13.225276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1314/2620] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:15.410791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:15.421881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ce884-9694-4063-984e-3e496c7997cb service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:15.427426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:15.428065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:15.428348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:15.428624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:15.430392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:15.442586 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:15.444168 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:15.456609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1307/2621] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:15.547005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:15.548436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:15.580165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5538526-6bc1-4e83-8095-c469d9f89cd9 service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:15.607671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:15.608727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:15.608994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:15.609313 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] PATH_INFO: `/v3/users/baac9b5e923747e6986720e3ac849056` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:15.610661 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baac9b5e923747e6986720e3ac849056)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:15.612617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:15.782678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baac9b5e923747e6986720e3ac849056'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:15.789909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baac9b5e923747e6986720e3ac849056'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:15.790812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baac9b5e923747e6986720e3ac849056'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:15.804263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baac9b5e923747e6986720e3ac849056'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:15.883551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7194114e-411c-48e2-9fd0-7e3c41f54703 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baac9b5e923747e6986720e3ac849056'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:15.890138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1315/2622] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:15 2022] DELETE /identity/v3/users/baac9b5e923747e6986720e3ac849056 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:16.153619 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:16.169027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f923f80-ce8d-4a95-a9e2-6442d81fe36c service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:16.227096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:16.227957 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.228280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.228555 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] PATH_INFO: `/v3/projects/af776ddc4c0f4e6390301a02c6bbfca0` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.229780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af776ddc4c0f4e6390301a02c6bbfca0)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:16.231691 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:16.346610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af776ddc4c0f4e6390301a02c6bbfca0', 'request_id': 'req-f00090f9-b937-4aa6-a6f4-48799f8f85c3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:16.421161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af776ddc4c0f4e6390301a02c6bbfca0', 'request_id': 'req-f00090f9-b937-4aa6-a6f4-48799f8f85c3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:16.428068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af776ddc4c0f4e6390301a02c6bbfca0', 'request_id': 'req-f00090f9-b937-4aa6-a6f4-48799f8f85c3'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:16.439570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] The token cache is being invalidate because project af776ddc4c0f4e6390301a02c6bbfca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:16.439862 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:16.440517 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-f00090f9-b937-4aa6-a6f4-48799f8f85c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:16.442439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1308/2623] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:16 2022] DELETE /identity/v3/projects/af776ddc4c0f4e6390301a02c6bbfca0 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:16.448478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1aecf-f202-464a-80a5-8ee1855e8dea None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.452750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1aecf-f202-464a-80a5-8ee1855e8dea None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.452750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1aecf-f202-464a-80a5-8ee1855e8dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.468785 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-8de1aecf-f202-464a-80a5-8ee1855e8dea None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:16.470293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-8de1aecf-f202-464a-80a5-8ee1855e8dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:16.532121 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1316/2624] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:16.540693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef948592-3cd0-46e6-a4f4-7275e16e5a7d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.541327 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef948592-3cd0-46e6-a4f4-7275e16e5a7d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.541617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef948592-3cd0-46e6-a4f4-7275e16e5a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.560997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-ef948592-3cd0-46e6-a4f4-7275e16e5a7d None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:16.562885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ef948592-3cd0-46e6-a4f4-7275e16e5a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:16.571192 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1309/2625] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:16.575800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1aecf-f202-464a-80a5-8ee1855e8dea None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:16.587102 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1aecf-f202-464a-80a5-8ee1855e8dea None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:16.592105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:16.592784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.593068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.593361 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.594094 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:48:16.594447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:16.596062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:16.598091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:16.609076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1317/2626] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:48:16.614428 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef948592-3cd0-46e6-a4f4-7275e16e5a7d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:16.625302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef948592-3cd0-46e6-a4f4-7275e16e5a7d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:16.629124 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:16.629816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.630100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.630377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.631109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:16.633028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:16.737606 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1310/2627] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:16 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:48:16.741308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:16.751105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067ef7e-29d3-422e-bfa6-ffcfee19bd7c admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:16.810392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:16.811066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.811346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.811621 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.812317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:16.814373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:16.913157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1318/2628] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:16 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:16.917296 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:16.928105 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f6702-3371-4fdd-97f4-f7812d3f0c93 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:16.932680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:16.933376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.937315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.937315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.937315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:16.937315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:16.947688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1311/2629] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:16.951551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:16.962826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f82854-1d80-4266-971d-e35f480cdc58 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:16.967872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b5399f-6162-43a1-98f6-158ae6dc2da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:16.968538 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5399f-6162-43a1-98f6-158ae6dc2da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:16.968826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5399f-6162-43a1-98f6-158ae6dc2da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:16.969109 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5399f-6162-43a1-98f6-158ae6dc2da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:16.969874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5399f-6162-43a1-98f6-158ae6dc2da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:16.971500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b5399f-6162-43a1-98f6-158ae6dc2da6 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:16.981809 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1319/2630] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:16.986129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:16.996280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fb865-afd5-4799-8401-dfc1e2c3ae8c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:17.001321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:17.002203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:17.002478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:17.002750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] PATH_INFO: `/v3/projects/107c9dab6bd24705b50f8d30181d6582/users/0e46c23c8e3946238795b800b921380c/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:17.018452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=107c9dab6bd24705b50f8d30181d6582, user_id=0e46c23c8e3946238795b800b921380c, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:17.021725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:17.142604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1312/2631] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:16 2022] PUT /identity/v3/projects/107c9dab6bd24705b50f8d30181d6582/users/0e46c23c8e3946238795b800b921380c/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:17.148875 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:17.149159 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:17.149474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:17.170021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:17.170021 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:17.234475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1320/2632] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:17.244847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:17.246846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:17.257736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:17.308331 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af894127-b0c6-451e-ac10-4213c87b9c12 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:17.312313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:17.312849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:17.313551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:17.313551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:17.324193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:17.325743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:17.335771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1313/2633] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:18.255504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:18.256907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:18.311754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd230d4-9a66-473b-9025-5ce9bdf06810 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:18.315241 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd641c1-e9ee-4504-b2a2-ed440a011b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:18.315874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd641c1-e9ee-4504-b2a2-ed440a011b54 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:18.316153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd641c1-e9ee-4504-b2a2-ed440a011b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:18.316426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd641c1-e9ee-4504-b2a2-ed440a011b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:18.318894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1dd641c1-e9ee-4504-b2a2-ed440a011b54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:18.347279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:18.384160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd641c1-e9ee-4504-b2a2-ed440a011b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:18.385870 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd641c1-e9ee-4504-b2a2-ed440a011b54 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:18.398655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1321/2634] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:18 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:18.840910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:18.842453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:18.855829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfdbfb-d4e5-4501-bbcd-30626999d29f service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:18.862687 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:18.863360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:18.863654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:18.864975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] PATH_INFO: `/v3/users/724673301f6549bdb23726232a962daf` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:18.865897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=724673301f6549bdb23726232a962daf)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:18.867859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:19.142968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '724673301f6549bdb23726232a962daf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:19.143637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '724673301f6549bdb23726232a962daf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:19.151045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '724673301f6549bdb23726232a962daf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:19.151707 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '724673301f6549bdb23726232a962daf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:19.165681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '724673301f6549bdb23726232a962daf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:19.266122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1314/2635] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:18 2022] DELETE /identity/v3/users/724673301f6549bdb23726232a962daf => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:19.271346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:19.275977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:19.275977 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:19.290333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:19.291525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:19.365737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1322/2636] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:19.375514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:19.376987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:19.471963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee16778-2709-46b2-ba77-4e1bef7e8601 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:19.478402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:19.479181 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:19.479593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:19.479972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:19.496366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:19.498204 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:19.512264 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1315/2637] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:48:20.148846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:20.150627 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:20.167966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d6f17-a080-48ff-8674-10791c314048 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:20.173010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:20.177470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:20.177470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:20.177470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] PATH_INFO: `/v3/projects/24859ed8e2ae4dc4bb93b8fc62d5aece` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:20.177470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24859ed8e2ae4dc4bb93b8fc62d5aece)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:20.178723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:20.298069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24859ed8e2ae4dc4bb93b8fc62d5aece', 'request_id': 'req-d3e25abb-c52b-4654-82eb-7312ea72c2c0'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.363526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24859ed8e2ae4dc4bb93b8fc62d5aece', 'request_id': 'req-d3e25abb-c52b-4654-82eb-7312ea72c2c0'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.372470 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24859ed8e2ae4dc4bb93b8fc62d5aece', 'request_id': 'req-d3e25abb-c52b-4654-82eb-7312ea72c2c0'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.388410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] The token cache is being invalidate because project 24859ed8e2ae4dc4bb93b8fc62d5aece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:20.388708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.389418 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.391226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1323/2638] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:20 2022] DELETE /identity/v3/projects/24859ed8e2ae4dc4bb93b8fc62d5aece => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:20.395930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:20.406768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271b28c-69cb-49fd-a985-4383c34687c5 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:20.431642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:20.490253 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:20.491740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:20.492965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:20.493514 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] PATH_INFO: `/v3/users/e68b0ad5ee124a5e9ebadd513351238c` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:20.495665 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e68b0ad5ee124a5e9ebadd513351238c)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:20.500445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:20.716951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e68b0ad5ee124a5e9ebadd513351238c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.717710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e68b0ad5ee124a5e9ebadd513351238c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.725622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e68b0ad5ee124a5e9ebadd513351238c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.726366 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e68b0ad5ee124a5e9ebadd513351238c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.742903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1fa8a3f8-0852-491e-8630-7afa4da435fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e68b0ad5ee124a5e9ebadd513351238c'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:20.838799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1316/2639] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:20 2022] DELETE /identity/v3/users/e68b0ad5ee124a5e9ebadd513351238c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:21.285244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:21.287382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:21.302855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e25abb-c52b-4654-82eb-7312ea72c2c0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:21.357996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:21.372272 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:21.374752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:21.374752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:21.374752 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] PATH_INFO: `/v3/projects/a05957387bb9499d83736eb09ab03c95` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:21.375463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a05957387bb9499d83736eb09ab03c95)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:21.377528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:21.512955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a05957387bb9499d83736eb09ab03c95', 'request_id': 'req-cb1f027d-585d-4f88-bb38-d826d85ad38d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:21.616994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a05957387bb9499d83736eb09ab03c95', 'request_id': 'req-cb1f027d-585d-4f88-bb38-d826d85ad38d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:21.628604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a05957387bb9499d83736eb09ab03c95', 'request_id': 'req-cb1f027d-585d-4f88-bb38-d826d85ad38d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:21.650564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] The token cache is being invalidate because project a05957387bb9499d83736eb09ab03c95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:21.651662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:21.653397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-cb1f027d-585d-4f88-bb38-d826d85ad38d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:21.658382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1324/2640] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:21 2022] DELETE /identity/v3/projects/a05957387bb9499d83736eb09ab03c95 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:21.664700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978afae-26e9-4a99-bfba-08752cfc426e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:21.665858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978afae-26e9-4a99-bfba-08752cfc426e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:21.666502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c978afae-26e9-4a99-bfba-08752cfc426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:21.687847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-c978afae-26e9-4a99-bfba-08752cfc426e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:21.689064 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c978afae-26e9-4a99-bfba-08752cfc426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:21.750820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1317/2641] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:21.759756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520dc00-6ee6-44e0-ad80-5a54281d41ea None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:21.760046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520dc00-6ee6-44e0-ad80-5a54281d41ea None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:21.760392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520dc00-6ee6-44e0-ad80-5a54281d41ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:21.781534 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7520dc00-6ee6-44e0-ad80-5a54281d41ea None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:21.782761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7520dc00-6ee6-44e0-ad80-5a54281d41ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:21.794634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1325/2642] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:21.796132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978afae-26e9-4a99-bfba-08752cfc426e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:21.810528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c978afae-26e9-4a99-bfba-08752cfc426e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:21.814704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:21.815637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:21.816024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:21.816310 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:21.817732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:48:21.818252 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:21.820096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:21.840082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1318/2643] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:48:21.842543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520dc00-6ee6-44e0-ad80-5a54281d41ea None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:21.854989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520dc00-6ee6-44e0-ad80-5a54281d41ea None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:21.868458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:21.869426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:21.869918 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:21.870332 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:21.871299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:21.873605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:21.974138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1326/2644] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:21 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:21.978100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:21.989952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12b141-d243-4e8c-98d2-8cf00fa3bbe7 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:22.023174 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:22.052903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:22.053620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.053909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.054190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.054890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:22.056803 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:22.185481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1319/2645] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:21 2022] POST /identity/v3/users => generated 284 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:22.188972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:22.201871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd778298-68c0-4d9a-9cfd-37539f2a467d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:22.209126 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:22.213236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.213236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.213236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.213236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:22.214512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:22.228077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1327/2646] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:22.231916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:22.246898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6c706-bd86-4c6f-a083-8c0485d09e07 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:22.256005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a1717-4831-4b31-9f86-39b976d46a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:22.257768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a1717-4831-4b31-9f86-39b976d46a13 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.257768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a1717-4831-4b31-9f86-39b976d46a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.257768 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a1717-4831-4b31-9f86-39b976d46a13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.258101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a1717-4831-4b31-9f86-39b976d46a13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:22.259642 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a1717-4831-4b31-9f86-39b976d46a13 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:22.272884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1320/2647] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:22.274175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:22.284707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095a5f-20e8-462d-8290-b07826784917 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:22.290778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:22.291682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.291967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.292311 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] PATH_INFO: `/v3/projects/980690d69ab14235a7505f33a310326c/users/53c64f4465e343d6a65a791529871f56/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.309425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980690d69ab14235a7505f33a310326c, user_id=53c64f4465e343d6a65a791529871f56, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:22.312203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:22.405524 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1328/2648] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:22 2022] PUT /identity/v3/projects/980690d69ab14235a7505f33a310326c/users/53c64f4465e343d6a65a791529871f56/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:22.417027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0b5a-3bcc-4ef7-8ae8-113910ccdca2 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.417335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0b5a-3bcc-4ef7-8ae8-113910ccdca2 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.417621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04f0b5a-3bcc-4ef7-8ae8-113910ccdca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.448949 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d04f0b5a-3bcc-4ef7-8ae8-113910ccdca2 None None] MFA Rules not processed for user `53c64f4465e343d6a65a791529871f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:22.450250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d04f0b5a-3bcc-4ef7-8ae8-113910ccdca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:22.533872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1321/2649] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:22.538956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:22.635129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581a1090-91d1-4725-9053-b8116ef11634 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:22.640605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237a0c-ca9c-41ea-b887-87c0214cc439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:22.641704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237a0c-ca9c-41ea-b887-87c0214cc439 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.642255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237a0c-ca9c-41ea-b887-87c0214cc439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.642745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237a0c-ca9c-41ea-b887-87c0214cc439 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.644273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c237a0c-ca9c-41ea-b887-87c0214cc439 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:22.646936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c237a0c-ca9c-41ea-b887-87c0214cc439 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:22.741310 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:22.753095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1329/2650] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:22 2022] POST /identity/v3/projects => generated 367 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:22.757843 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f0b5a-3bcc-4ef7-8ae8-113910ccdca2 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:22.771088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04f0b5a-3bcc-4ef7-8ae8-113910ccdca2 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:22.849436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.849614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.849871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.886057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:22.886888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:22.887297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:22.887679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:22.888604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:22.891207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:22.901359 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] MFA Rules not processed for user `0e46c23c8e3946238795b800b921380c`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:22.903212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:22.974222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1330/2651] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:48:22.989976 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:22.991440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:23.006329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1322/2652] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:22 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:23.010709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:23.022860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8198b7-621c-43c5-bec7-116275b7bc69 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:23.027634 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:23.028313 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:23.029943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:23.029943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:23.029943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:23.031519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:23.042992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1323/2653] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:23.047496 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:23.058866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71fddb-4623-4662-9055-02669a6ad50a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:23.068474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:23.069413 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:23.069699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:23.069975 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] PATH_INFO: `/v3/projects/fd2721c8a5de432eac7bbf2f51882248/users/90b13362bd0b46bcb8f2ef79a1551b68/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:23.080805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023870d6-6425-4916-a199-5f87563b8d07 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:23.089462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2721c8a5de432eac7bbf2f51882248, user_id=90b13362bd0b46bcb8f2ef79a1551b68, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:23.090436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:23.091247 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:23.091539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:23.091983 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:23.092211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:23.095903 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:23.102209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:23.103975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:23.114271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1331/2654] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:48:23.164329 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1324/2655] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:23 2022] PUT /identity/v3/projects/fd2721c8a5de432eac7bbf2f51882248/users/90b13362bd0b46bcb8f2ef79a1551b68/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:23.173433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:23.178915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8b359-94aa-4388-b25b-71ef9a10593a service nova] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:23.243297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694d6a5-d3ba-4a09-b126-14db39a167e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:23.244011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694d6a5-d3ba-4a09-b126-14db39a167e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:23.244291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694d6a5-d3ba-4a09-b126-14db39a167e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:23.244568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694d6a5-d3ba-4a09-b126-14db39a167e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:23.245330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694d6a5-d3ba-4a09-b126-14db39a167e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:23.246947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694d6a5-d3ba-4a09-b126-14db39a167e9 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:23.258501 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1332/2656] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:23.262610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:23.273522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be97a4f8-f9d0-47a2-af53-29502e7f70cf admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:23.278341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:23.279237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:23.289488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:23.289488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] PATH_INFO: `/v3/projects/fd2721c8a5de432eac7bbf2f51882248/users/90b13362bd0b46bcb8f2ef79a1551b68/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:23.289488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd2721c8a5de432eac7bbf2f51882248, user_id=90b13362bd0b46bcb8f2ef79a1551b68, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:23.290454 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:23.310323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1325/2657] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:23 2022] PUT /identity/v3/projects/fd2721c8a5de432eac7bbf2f51882248/users/90b13362bd0b46bcb8f2ef79a1551b68/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:23.322216 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35118bc2-2c78-4e9e-af4d-1289de267eca None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:23.322731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35118bc2-2c78-4e9e-af4d-1289de267eca None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:23.322855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35118bc2-2c78-4e9e-af4d-1289de267eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:23.343357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=100 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:23.354197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-35118bc2-2c78-4e9e-af4d-1289de267eca None None] MFA Rules not processed for user `90b13362bd0b46bcb8f2ef79a1551b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:23.355530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-35118bc2-2c78-4e9e-af4d-1289de267eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:23.441544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1333/2658] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:23 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:23.449969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:23.451869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:23.515259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c838b625-2706-445a-8554-aa62615c0aa0 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:23.518981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:23.520173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:23.520510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:23.521009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:23.532227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:23.533865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:23.553316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1326/2659] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:23 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:26.264197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243796e-dfeb-48a7-99ba-0b93d25c998b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:26.264745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243796e-dfeb-48a7-99ba-0b93d25c998b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:26.264745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5243796e-dfeb-48a7-99ba-0b93d25c998b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:26.288306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-5243796e-dfeb-48a7-99ba-0b93d25c998b None None] MFA Rules not processed for user `0e46c23c8e3946238795b800b921380c`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:26.289565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5243796e-dfeb-48a7-99ba-0b93d25c998b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:26.372365 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1334/2660] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:27.018758 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:27.119294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d4f20d-ac5a-46c5-b027-92357449ffb2 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:27.123877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:27.137335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:27.137335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:27.137335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] PATH_INFO: `/v3/users/53c64f4465e343d6a65a791529871f56` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:27.137335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c64f4465e343d6a65a791529871f56)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:27.137335 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:27.399063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c64f4465e343d6a65a791529871f56'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:27.399760 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c64f4465e343d6a65a791529871f56'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:27.406726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c64f4465e343d6a65a791529871f56'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:27.407402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c64f4465e343d6a65a791529871f56'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:27.419784 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c64f4465e343d6a65a791529871f56'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:27.512001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1327/2661] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:27 2022] DELETE /identity/v3/users/53c64f4465e343d6a65a791529871f56 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:27.516422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:27.516920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:27.517023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:27.546036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:27.547396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:27.637135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1335/2662] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:48:27.648492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:27.649964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:27.655458 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:27.709902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c2898-24a8-40c0-9f78-3edd00264622 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:27.714219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:27.716222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:27.716222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:27.716222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:27.725250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:27.726715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:27.736498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1328/2663] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:28.130360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:28.131839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:28.144078 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db859dbe-63ee-48f0-aad7-ef02dc47927f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:28.152650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:28.153655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:28.153940 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:28.154215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] PATH_INFO: `/v3/projects/980690d69ab14235a7505f33a310326c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:28.155594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980690d69ab14235a7505f33a310326c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:28.161650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:28.263555 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:28.273526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980690d69ab14235a7505f33a310326c', 'request_id': 'req-96a5a5bc-1a4a-49b5-9281-509fb7738de8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.351297 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980690d69ab14235a7505f33a310326c', 'request_id': 'req-96a5a5bc-1a4a-49b5-9281-509fb7738de8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.358106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980690d69ab14235a7505f33a310326c', 'request_id': 'req-96a5a5bc-1a4a-49b5-9281-509fb7738de8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.368794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] The token cache is being invalidate because project 980690d69ab14235a7505f33a310326c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:28.369077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.369760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.371345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1336/2664] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:28 2022] DELETE /identity/v3/projects/980690d69ab14235a7505f33a310326c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:28.375255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:28.385510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6971f607-778e-4a27-b0f9-1e4fbfd97707 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:28.441184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:28.442173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:28.442898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:28.443175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] PATH_INFO: `/v3/users/90b13362bd0b46bcb8f2ef79a1551b68` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:28.444545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b13362bd0b46bcb8f2ef79a1551b68)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:28.446494 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:28.680113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b13362bd0b46bcb8f2ef79a1551b68'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.680781 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b13362bd0b46bcb8f2ef79a1551b68'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.687800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b13362bd0b46bcb8f2ef79a1551b68'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.688421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b13362bd0b46bcb8f2ef79a1551b68'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.700610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-dc83b8b5-fdf2-45c5-a499-69f6fe27303e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b13362bd0b46bcb8f2ef79a1551b68'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:28.797543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1329/2665] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:28 2022] DELETE /identity/v3/users/90b13362bd0b46bcb8f2ef79a1551b68 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:48:29.177292 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:29.177804 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:29.193907 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5a5bc-1a4a-49b5-9281-509fb7738de8 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:29.251657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:29.252537 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.252806 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.253085 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] PATH_INFO: `/v3/projects/fd2721c8a5de432eac7bbf2f51882248` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.254318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd2721c8a5de432eac7bbf2f51882248)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:29.256953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:29.392413 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd2721c8a5de432eac7bbf2f51882248', 'request_id': 'req-e38c0330-3f39-44d5-848a-e6589b7f7c88'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:29.460103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd2721c8a5de432eac7bbf2f51882248', 'request_id': 'req-e38c0330-3f39-44d5-848a-e6589b7f7c88'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:29.467143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd2721c8a5de432eac7bbf2f51882248', 'request_id': 'req-e38c0330-3f39-44d5-848a-e6589b7f7c88'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:29.478077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] The token cache is being invalidate because project fd2721c8a5de432eac7bbf2f51882248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:29.478368 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:29.479020 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-e38c0330-3f39-44d5-848a-e6589b7f7c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:29.480574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1337/2666] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:29 2022] DELETE /identity/v3/projects/fd2721c8a5de432eac7bbf2f51882248 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:29.485507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcc5d9-3217-45cf-97f9-dd4df39601e8 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.485981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcc5d9-3217-45cf-97f9-dd4df39601e8 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.486113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcc5d9-3217-45cf-97f9-dd4df39601e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.506045 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-bafcc5d9-3217-45cf-97f9-dd4df39601e8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:29.507263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-bafcc5d9-3217-45cf-97f9-dd4df39601e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:29.526631 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:29.568447 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1330/2667] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:29.576260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e99f69-f5d6-40cf-bc6d-759159d68431 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.576552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e99f69-f5d6-40cf-bc6d-759159d68431 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.576835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e99f69-f5d6-40cf-bc6d-759159d68431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.595357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-30e99f69-f5d6-40cf-bc6d-759159d68431 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:29.596579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-30e99f69-f5d6-40cf-bc6d-759159d68431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:29.605187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1338/2668] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:29.609550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcc5d9-3217-45cf-97f9-dd4df39601e8 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:29.619478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcc5d9-3217-45cf-97f9-dd4df39601e8 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:29.624088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:29.624738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.625024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.625340 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.626066 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:48:29.626381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:29.627971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:29.640135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1331/2669] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:48:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:48:29.645992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e99f69-f5d6-40cf-bc6d-759159d68431 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:29.655206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e99f69-f5d6-40cf-bc6d-759159d68431 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:29.660203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:29.660948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.661516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.661946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.662656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:29.664655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:29.748026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1339/2670] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:29 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:29.755982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:29.766342 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0f9e1-f6a4-4f4f-98f5-23000368e16a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:29.823069 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:29.823815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.824150 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.824431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.825118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:29.827035 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:29.894529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1332/2671] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:29 2022] POST /identity/v3/users => generated 292 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:29.899295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:29.909778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f4af0-d8ff-4ff2-8a31-36f4603b1aa1 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:29.914645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:29.915852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.915852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.915852 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.916721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:29.918417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:29.929333 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1340/2672] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:29.933401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:29.944438 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65438e33-75e2-4687-8306-2b52d217370d admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:29.949034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5468a-6922-4ca3-a6a5-ff1777d2f88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:29.949736 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5468a-6922-4ca3-a6a5-ff1777d2f88d admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.950072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5468a-6922-4ca3-a6a5-ff1777d2f88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.950467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5468a-6922-4ca3-a6a5-ff1777d2f88d admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.951200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5468a-6922-4ca3-a6a5-ff1777d2f88d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:29.952788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5468a-6922-4ca3-a6a5-ff1777d2f88d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:29.963726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1333/2673] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:29.968005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:29.978570 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc31a9-2854-4782-af7f-f6eaad354db7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:29.983304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:29.984189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:29.984462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:29.984864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] PATH_INFO: `/v3/projects/51173a2b2a3240ffae550483a298e767/users/0f378a1532304b929471199b9e7f860b/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:29.997082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:30.001145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51173a2b2a3240ffae550483a298e767, user_id=0f378a1532304b929471199b9e7f860b, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:30.007869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:30.076790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1341/2674] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:29 2022] PUT /identity/v3/projects/51173a2b2a3240ffae550483a298e767/users/0f378a1532304b929471199b9e7f860b/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:30.087830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241ef1b-cf87-443e-8355-fcb54195802d None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.088122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241ef1b-cf87-443e-8355-fcb54195802d None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.088401 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5241ef1b-cf87-443e-8355-fcb54195802d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.115167 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5241ef1b-cf87-443e-8355-fcb54195802d None None] MFA Rules not processed for user `0f378a1532304b929471199b9e7f860b`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:30.116384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5241ef1b-cf87-443e-8355-fcb54195802d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:30.183128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1334/2675] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:30.187669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:30.248480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f2519d-7622-4706-a2a3-9beb8a29244b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:30.253221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:30.253980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.254263 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.254586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.255298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:30.257192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:30.352987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1342/2676] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:30 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:48:30.360778 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241ef1b-cf87-443e-8355-fcb54195802d None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:30.372077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5241ef1b-cf87-443e-8355-fcb54195802d None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:30.390087 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:30.434880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:30.435620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.435910 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.436190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.436881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:30.438844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:30.530262 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1335/2677] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:30 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:30.534279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:30.545035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c7dfe-3704-46f6-8c5d-d80bdbca1157 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:30.549830 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:30.550506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.550787 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.551066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.552124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:30.553812 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:30.565206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1343/2678] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:30.569902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:30.580590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e3e20-cc47-4752-9152-3e3cdcde66fb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:30.586006 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:30.586905 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.587189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.587466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] PATH_INFO: `/v3/projects/ae84c869691a4d9ebb5d5c37d0264950/users/3f2cea8737624d8bb0493b6a6ee781f1/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.604439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae84c869691a4d9ebb5d5c37d0264950, user_id=3f2cea8737624d8bb0493b6a6ee781f1, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:30.610384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:30.666752 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1336/2679] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:30 2022] PUT /identity/v3/projects/ae84c869691a4d9ebb5d5c37d0264950/users/3f2cea8737624d8bb0493b6a6ee781f1/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:30.670678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:30.685562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54402e3-4b9b-4ccb-9553-b0e68198d441 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:30.745582 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf43771-4f62-48e7-858d-b06f32117661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:30.746266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43771-4f62-48e7-858d-b06f32117661 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.746685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43771-4f62-48e7-858d-b06f32117661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.746985 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43771-4f62-48e7-858d-b06f32117661 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.747709 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf43771-4f62-48e7-858d-b06f32117661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:30.749331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf43771-4f62-48e7-858d-b06f32117661 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:30.762684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1344/2680] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:30 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:30.765568 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:30.780357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767aa0b-319c-44a1-b063-dbbf15fee617 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:30.785397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:30.787320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.788230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.788230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] PATH_INFO: `/v3/projects/ae84c869691a4d9ebb5d5c37d0264950/users/3f2cea8737624d8bb0493b6a6ee781f1/roles/e1e5746208514068acadbab89907c473` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.790245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae84c869691a4d9ebb5d5c37d0264950, user_id=3f2cea8737624d8bb0493b6a6ee781f1, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:30.792950 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:30.804627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1337/2681] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:30 2022] PUT /identity/v3/projects/ae84c869691a4d9ebb5d5c37d0264950/users/3f2cea8737624d8bb0493b6a6ee781f1/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:48:30.815882 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c2f32-7095-4bf2-8f2a-215001dba5d7 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:30.816169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c2f32-7095-4bf2-8f2a-215001dba5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:30.817006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c2f32-7095-4bf2-8f2a-215001dba5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:30.854344 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-494c2f32-7095-4bf2-8f2a-215001dba5d7 None None] MFA Rules not processed for user `3f2cea8737624d8bb0493b6a6ee781f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:30.855574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-494c2f32-7095-4bf2-8f2a-215001dba5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:30.916990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:30.939139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1345/2682] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:30.950170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:30.952291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:31.014604 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24cb59-0788-48d8-88d4-756dda829072 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:31.018339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:31.019011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:31.019298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:31.019577 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:31.037190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:31.037190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:31.045396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1338/2683] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:48:31.170243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c2f32-7095-4bf2-8f2a-215001dba5d7 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:31.177443 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c2f32-7095-4bf2-8f2a-215001dba5d7 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:31.181634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:31.182283 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:31.182571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:31.182848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:31.251909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:31.253481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:31.268209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1346/2684] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:32.272385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:32.275582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:32.356011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7283227c-db24-4fd7-ac31-eba2e4c2ec42 service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:32.361660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c4da4-63b3-4a7b-b679-36981c0b6541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:32.362318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c4da4-63b3-4a7b-b679-36981c0b6541 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:32.362727 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c4da4-63b3-4a7b-b679-36981c0b6541 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:32.362855 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c4da4-63b3-4a7b-b679-36981c0b6541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:32.365565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ef5c4da4-63b3-4a7b-b679-36981c0b6541 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:32.378724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c4da4-63b3-4a7b-b679-36981c0b6541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:32.381249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c4da4-63b3-4a7b-b679-36981c0b6541 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:32.388162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:32.394783 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1339/2685] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:36.529775 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:36.593760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d42fed-db5a-4169-bc2c-bd9b25a9ae99 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:36.598349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:36.599136 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:36.599421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:36.599697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] PATH_INFO: `/v3/users/0f378a1532304b929471199b9e7f860b` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:36.600899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f378a1532304b929471199b9e7f860b)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:36.602865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:36.756064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f378a1532304b929471199b9e7f860b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:36.763207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f378a1532304b929471199b9e7f860b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:36.763880 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f378a1532304b929471199b9e7f860b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:36.776718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f378a1532304b929471199b9e7f860b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:36.825043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f378a1532304b929471199b9e7f860b'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:36.831259 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1347/2686] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:36 2022] DELETE /identity/v3/users/0f378a1532304b929471199b9e7f860b => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:36.836070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:36.836546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:36.836648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:36.854272 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:36.855381 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:36.912349 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1340/2687] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:36.922169 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:36.923564 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:36.929479 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:36.981819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6311a548-46a2-4503-80cc-0949098a46ef admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:36.986048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:36.986695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:36.986978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:36.987250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:36.996924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:36.998416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:37.008389 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1348/2688] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:37.392187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:37.394122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:37.408283 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5285ee22-7368-4379-bbcd-7111a70eb663 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:37.413805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:37.414615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:37.414896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:37.415169 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] PATH_INFO: `/v3/projects/51173a2b2a3240ffae550483a298e767` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:37.416344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51173a2b2a3240ffae550483a298e767)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:37.418311 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:37.512747 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51173a2b2a3240ffae550483a298e767', 'request_id': 'req-86c865f3-2999-4063-900c-0ea9543ffad4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.562087 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51173a2b2a3240ffae550483a298e767', 'request_id': 'req-86c865f3-2999-4063-900c-0ea9543ffad4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.570554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51173a2b2a3240ffae550483a298e767', 'request_id': 'req-86c865f3-2999-4063-900c-0ea9543ffad4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.581238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] The token cache is being invalidate because project 51173a2b2a3240ffae550483a298e767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:37.581556 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.582194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.587785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1341/2689] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:37 2022] DELETE /identity/v3/projects/51173a2b2a3240ffae550483a298e767 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:37.589174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:37.600520 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667daab-a9c3-4820-b92a-5c68cbaa31de service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:37.659311 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:37.660195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:37.661140 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:37.661453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] PATH_INFO: `/v3/users/3f2cea8737624d8bb0493b6a6ee781f1` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:37.662760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2cea8737624d8bb0493b6a6ee781f1)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:37.664755 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:37.805171 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2cea8737624d8bb0493b6a6ee781f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.812676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2cea8737624d8bb0493b6a6ee781f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.813395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2cea8737624d8bb0493b6a6ee781f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.826325 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2cea8737624d8bb0493b6a6ee781f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.917718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ec45939c-495b-4c7d-9754-afee7bf9d19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2cea8737624d8bb0493b6a6ee781f1'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:37.924926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1349/2690] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:37 2022] DELETE /identity/v3/users/3f2cea8737624d8bb0493b6a6ee781f1 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:38.369754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:38.371328 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:38.385149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c865f3-2999-4063-900c-0ea9543ffad4 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:38.443246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:38.444113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:38.444395 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:38.444671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] PATH_INFO: `/v3/projects/ae84c869691a4d9ebb5d5c37d0264950` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:38.445924 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae84c869691a4d9ebb5d5c37d0264950)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:38.447831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:38.569572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:38.586115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae84c869691a4d9ebb5d5c37d0264950', 'request_id': 'req-24915d27-0207-4144-ae0e-8a977e3e2721'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:38.670688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae84c869691a4d9ebb5d5c37d0264950', 'request_id': 'req-24915d27-0207-4144-ae0e-8a977e3e2721'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:38.676749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae84c869691a4d9ebb5d5c37d0264950', 'request_id': 'req-24915d27-0207-4144-ae0e-8a977e3e2721'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:38.690189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] The token cache is being invalidate because project ae84c869691a4d9ebb5d5c37d0264950 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:38.690486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:38.691144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-24915d27-0207-4144-ae0e-8a977e3e2721 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:38.693289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1342/2691] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:38 2022] DELETE /identity/v3/projects/ae84c869691a4d9ebb5d5c37d0264950 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:38.698589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d048b-4187-41c9-9dd7-43797c8e231a None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:38.703082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d048b-4187-41c9-9dd7-43797c8e231a None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:38.703082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d048b-4187-41c9-9dd7-43797c8e231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:38.721245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-911d048b-4187-41c9-9dd7-43797c8e231a None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:38.722490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-911d048b-4187-41c9-9dd7-43797c8e231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:38.742674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:38.784234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1350/2692] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:38.792849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864df2-ab9b-4832-a94b-b69306cfe770 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:38.793142 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864df2-ab9b-4832-a94b-b69306cfe770 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:38.793468 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe864df2-ab9b-4832-a94b-b69306cfe770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:38.812247 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-fe864df2-ab9b-4832-a94b-b69306cfe770 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:38.815896 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fe864df2-ab9b-4832-a94b-b69306cfe770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:38.822250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1343/2693] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:38.826375 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d048b-4187-41c9-9dd7-43797c8e231a None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:38.836544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d048b-4187-41c9-9dd7-43797c8e231a None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:38.841399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:38.842050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:38.842331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:38.842606 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:38.843335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:48:38.843637 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:38.845233 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:38.857035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1351/2694] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:48:38.861502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864df2-ab9b-4832-a94b-b69306cfe770 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:38.870931 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe864df2-ab9b-4832-a94b-b69306cfe770 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:38.875486 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:38.876128 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:38.876408 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:38.876680 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:38.877403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:38.879834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:38.993423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1344/2695] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:38 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:38.997613 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.008300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a59ee1-a463-4fff-97d1-a1646cebea8c admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.067605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.068384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.068665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.068941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.069777 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:39.071754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:39.215135 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1352/2696] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:38 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:48:39.219344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.229668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772b9be-3c6d-40a1-a201-301eda97e745 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.234148 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.234850 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.240410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.240410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.240410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:39.240410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:39.247983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1345/2697] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:39.252101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.262992 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d2357-bf32-40b9-bf48-3705726880b1 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.267634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637b7f52-30ee-48a6-b774-4cc116cc5783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.268294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b7f52-30ee-48a6-b774-4cc116cc5783 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.268581 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b7f52-30ee-48a6-b774-4cc116cc5783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.268860 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637b7f52-30ee-48a6-b774-4cc116cc5783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.269650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b7f52-30ee-48a6-b774-4cc116cc5783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:39.274869 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637b7f52-30ee-48a6-b774-4cc116cc5783 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:39.281423 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1353/2698] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:39.285935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.295763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db69f1-9762-4044-b76a-faf5ce2a34a5 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.300673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.302242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.302535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.302814 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] PATH_INFO: `/v3/projects/bbdf508c56c541b49e53089140b38ce2/users/f666405ecd194a6ebc562aaef32baebd/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.317860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbdf508c56c541b49e53089140b38ce2, user_id=f666405ecd194a6ebc562aaef32baebd, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:39.320523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:39.408884 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1346/2699] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:39 2022] PUT /identity/v3/projects/bbdf508c56c541b49e53089140b38ce2/users/f666405ecd194a6ebc562aaef32baebd/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:39.420056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bfafe-fd07-4d2b-9fb0-a626bcf40f65 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.420373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bfafe-fd07-4d2b-9fb0-a626bcf40f65 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.420653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bfafe-fd07-4d2b-9fb0-a626bcf40f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.448730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4f5bfafe-fd07-4d2b-9fb0-a626bcf40f65 None None] MFA Rules not processed for user `f666405ecd194a6ebc562aaef32baebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:39.449901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4f5bfafe-fd07-4d2b-9fb0-a626bcf40f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:39.515546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1354/2700] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:39.520078 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.593869 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7dec08-c022-4a18-a9a2-8a176230271b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.599866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.600678 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.601037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.601417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.602396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:39.604453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:39.737477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1347/2701] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:39 2022] POST /identity/v3/projects => generated 365 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:39.740565 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bfafe-fd07-4d2b-9fb0-a626bcf40f65 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.751489 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bfafe-fd07-4d2b-9fb0-a626bcf40f65 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.768584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:39.815735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.816429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.816832 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.817229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.818130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:39.820188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:39.928138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1355/2702] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:39 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:39.932238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.943505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b304ed-0b6c-4b48-a3a7-f8f046b3a74a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.949377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.949987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.950275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.950557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:39.951293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:39.952906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:39.964807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1348/2703] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:39.969084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:39.980129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f5988-2192-4868-b0ec-54a69502e848 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:39.985033 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:39.986068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:39.986353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:39.986629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] PATH_INFO: `/v3/projects/8a517ccf24ca4cea965ef4411db7c095/users/27972a2ffd4f4a02ae485863201f1308/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:40.002851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a517ccf24ca4cea965ef4411db7c095, user_id=27972a2ffd4f4a02ae485863201f1308, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:40.005708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:40.105901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1356/2704] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:39 2022] PUT /identity/v3/projects/8a517ccf24ca4cea965ef4411db7c095/users/27972a2ffd4f4a02ae485863201f1308/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:40.109984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:40.120214 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9008fa3-84ba-486c-88b1-8461eea0b68b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:40.142355 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:40.181902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1460a-182e-4643-8834-c509131cec5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:40.185867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1460a-182e-4643-8834-c509131cec5d admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:40.185867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1460a-182e-4643-8834-c509131cec5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:40.185867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1460a-182e-4643-8834-c509131cec5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:40.185867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1460a-182e-4643-8834-c509131cec5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:40.185867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1460a-182e-4643-8834-c509131cec5d admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:40.196877 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1349/2705] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:40.201205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:40.212927 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d684f1f-e432-4017-bda9-68a664db373d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:40.218504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:40.221080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:40.221080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:40.221080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] PATH_INFO: `/v3/projects/8a517ccf24ca4cea965ef4411db7c095/users/27972a2ffd4f4a02ae485863201f1308/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:40.222105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a517ccf24ca4cea965ef4411db7c095, user_id=27972a2ffd4f4a02ae485863201f1308, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:40.225080 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:40.245844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1357/2706] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:40 2022] PUT /identity/v3/projects/8a517ccf24ca4cea965ef4411db7c095/users/27972a2ffd4f4a02ae485863201f1308/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:40.251711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2652-76b8-430c-a4a8-26fc978dadad None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:40.251711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2652-76b8-430c-a4a8-26fc978dadad None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:40.251711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec2652-76b8-430c-a4a8-26fc978dadad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:40.279963 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f8ec2652-76b8-430c-a4a8-26fc978dadad None None] MFA Rules not processed for user `27972a2ffd4f4a02ae485863201f1308`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:40.281231 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f8ec2652-76b8-430c-a4a8-26fc978dadad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:40.352269 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1350/2707] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:48:40.375151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:40.376827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:40.435019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8033d9-61d2-4896-b119-b556f4a9b6af admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:40.438656 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:40.439302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:40.439622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:40.439897 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:40.450178 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:40.451624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:40.462554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1358/2708] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:40.669120 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2652-76b8-430c-a4a8-26fc978dadad None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:40.679361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec2652-76b8-430c-a4a8-26fc978dadad None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:40.682977 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c531b-122c-4014-9034-da9b0bb160ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:40.683610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c531b-122c-4014-9034-da9b0bb160ce service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:40.683889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c531b-122c-4014-9034-da9b0bb160ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:40.684159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c531b-122c-4014-9034-da9b0bb160ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:40.740908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c531b-122c-4014-9034-da9b0bb160ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:40.742362 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c531b-122c-4014-9034-da9b0bb160ce service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:40.751081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1351/2709] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:47.170865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:47.234412 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6cc4d7-acfa-4bb3-9d94-f487b94d5e57 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:47.239318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:47.240194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:47.240473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:47.240747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] PATH_INFO: `/v3/users/f666405ecd194a6ebc562aaef32baebd` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:47.241978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f666405ecd194a6ebc562aaef32baebd)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:47.243913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:47.260987 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:47.501206 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f666405ecd194a6ebc562aaef32baebd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:47.508530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f666405ecd194a6ebc562aaef32baebd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:47.509219 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f666405ecd194a6ebc562aaef32baebd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:47.521996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f666405ecd194a6ebc562aaef32baebd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:47.567396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f666405ecd194a6ebc562aaef32baebd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:47.575786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1359/2710] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:47 2022] DELETE /identity/v3/users/f666405ecd194a6ebc562aaef32baebd => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:47.579816 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:47.580280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:47.580387 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:47.598425 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:47.599644 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:47.640550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=100 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:47.660005 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1352/2711] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:47.669792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:47.671323 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:47.728270 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df344-6363-4fa5-b7d5-97684701b090 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:47.731900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:47.732685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:47.732963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:47.733237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:47.742566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:47.743989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:47.753449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1360/2712] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:48.209948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:48.211838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:48.224473 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5ff3a0-3fac-4605-9eb9-57fd6d1bbdfa None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:48.235130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:48.236031 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:48.236319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:48.236599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] PATH_INFO: `/v3/projects/bbdf508c56c541b49e53089140b38ce2` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:48.238084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbdf508c56c541b49e53089140b38ce2)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:48.240058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:48.373833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbdf508c56c541b49e53089140b38ce2', 'request_id': 'req-339fee28-08fc-4c23-b8b8-e87c6875cf54'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.430115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbdf508c56c541b49e53089140b38ce2', 'request_id': 'req-339fee28-08fc-4c23-b8b8-e87c6875cf54'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.436086 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbdf508c56c541b49e53089140b38ce2', 'request_id': 'req-339fee28-08fc-4c23-b8b8-e87c6875cf54'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.447858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] The token cache is being invalidate because project bbdf508c56c541b49e53089140b38ce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:48.448149 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.448780 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.450365 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1353/2713] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:48 2022] DELETE /identity/v3/projects/bbdf508c56c541b49e53089140b38ce2 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:48.454405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:48.464948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c3e47-5086-4c25-a45b-1b1169a60c2e service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:48.521328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:48.522769 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:48.523052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:48.523327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] PATH_INFO: `/v3/users/27972a2ffd4f4a02ae485863201f1308` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:48.524509 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27972a2ffd4f4a02ae485863201f1308)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:48.526460 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:48.686318 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:48.691982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27972a2ffd4f4a02ae485863201f1308'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.699674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27972a2ffd4f4a02ae485863201f1308'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.700343 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27972a2ffd4f4a02ae485863201f1308'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.712315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27972a2ffd4f4a02ae485863201f1308'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.796427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-35413d0f-86dd-4a60-8e67-1cca4e189c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27972a2ffd4f4a02ae485863201f1308'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:48.803526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1361/2714] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:48 2022] DELETE /identity/v3/users/27972a2ffd4f4a02ae485863201f1308 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:49.192076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:49.194152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:49.210804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fee28-08fc-4c23-b8b8-e87c6875cf54 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:49.265046 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:49.265939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.266227 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.266503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] PATH_INFO: `/v3/projects/8a517ccf24ca4cea965ef4411db7c095` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.267674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a517ccf24ca4cea965ef4411db7c095)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:49.269616 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:49.368611 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a517ccf24ca4cea965ef4411db7c095', 'request_id': 'req-6041fbd6-bafc-4770-a11d-c3697b02ccf4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:49.435565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a517ccf24ca4cea965ef4411db7c095', 'request_id': 'req-6041fbd6-bafc-4770-a11d-c3697b02ccf4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:49.441386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a517ccf24ca4cea965ef4411db7c095', 'request_id': 'req-6041fbd6-bafc-4770-a11d-c3697b02ccf4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:49.449407 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:49.454048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] The token cache is being invalidate because project 8a517ccf24ca4cea965ef4411db7c095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:49.454338 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:49.454968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-6041fbd6-bafc-4770-a11d-c3697b02ccf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:49.456545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1354/2715] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:49 2022] DELETE /identity/v3/projects/8a517ccf24ca4cea965ef4411db7c095 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:49.461901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d9561-1da2-4a65-842c-1471ec7aa44f None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.462650 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d9561-1da2-4a65-842c-1471ec7aa44f None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.462778 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d9561-1da2-4a65-842c-1471ec7aa44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.480139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-606d9561-1da2-4a65-842c-1471ec7aa44f None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:49.481367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-606d9561-1da2-4a65-842c-1471ec7aa44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:49.542494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1362/2716] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:48:49.550442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb16c2-8246-4877-a126-fc30d9311a6e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.550737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb16c2-8246-4877-a126-fc30d9311a6e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.551018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb16c2-8246-4877-a126-fc30d9311a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.569847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-7bcb16c2-8246-4877-a126-fc30d9311a6e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:49.574040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7bcb16c2-8246-4877-a126-fc30d9311a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:49.582050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1355/2717] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:48:49.586234 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d9561-1da2-4a65-842c-1471ec7aa44f None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:49.596104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d9561-1da2-4a65-842c-1471ec7aa44f None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:49.600610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:49.601301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.601754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.602005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.606826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:48:49.606826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:49.606826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:49.616238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1363/2718] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:48:49.621015 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb16c2-8246-4877-a126-fc30d9311a6e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:49.630237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb16c2-8246-4877-a126-fc30d9311a6e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:49.635601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:49.636251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.636537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.636815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.637533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:49.639426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:49.729298 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1356/2719] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:49 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:49.733410 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:49.744130 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1651ab0d-bd40-439a-9731-0e11e76ee549 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:49.798848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:49.799513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.799799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.800076 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.800759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:49.802694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:49.870593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1364/2720] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:49 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:49.874629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:49.885424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec755c-a1bf-4856-8eca-e62425adf9fc admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:49.890025 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:49.890682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.890971 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.891254 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.891983 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:49.893624 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:49.904159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1357/2721] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:49.908107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:49.918871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfccea8a-bdf5-4d6a-a94c-10235d399160 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:49.923605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d8ada-17b3-4b59-96c5-71073c5f9c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:49.924261 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d8ada-17b3-4b59-96c5-71073c5f9c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.924548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d8ada-17b3-4b59-96c5-71073c5f9c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.924826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d8ada-17b3-4b59-96c5-71073c5f9c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.925593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d8ada-17b3-4b59-96c5-71073c5f9c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:49.927182 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d8ada-17b3-4b59-96c5-71073c5f9c90 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:49.938225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1365/2722] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:49.942052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:49.952223 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e09770-8a61-4df9-a33b-b6b669eed375 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:49.956788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:49.957700 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:49.957988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:49.958271 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] PATH_INFO: `/v3/projects/691b58b096fa43c49da6eaf9a6e7de36/users/ceac655fafcd4843abf0a290b412dfe2/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:49.974316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=691b58b096fa43c49da6eaf9a6e7de36, user_id=ceac655fafcd4843abf0a290b412dfe2, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:49.977464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:50.029820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1358/2723] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:49 2022] PUT /identity/v3/projects/691b58b096fa43c49da6eaf9a6e7de36/users/ceac655fafcd4843abf0a290b412dfe2/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:50.042245 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbea93-151a-4f8a-9785-4c433ffa7b59 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:50.042540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbea93-151a-4f8a-9785-4c433ffa7b59 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:50.042823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbea93-151a-4f8a-9785-4c433ffa7b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:50.069346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-edcbea93-151a-4f8a-9785-4c433ffa7b59 None None] MFA Rules not processed for user `ceac655fafcd4843abf0a290b412dfe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:50.070583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-edcbea93-151a-4f8a-9785-4c433ffa7b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:50.079547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:50.135306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1366/2724] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:50.147267 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:50.149620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:50.206256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ee8f8-eb4d-4763-8fd7-dab86b14d2e8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:50.209849 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe3855-2a41-4dbe-a13c-9c1118d56223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:50.210498 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3855-2a41-4dbe-a13c-9c1118d56223 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:50.211325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3855-2a41-4dbe-a13c-9c1118d56223 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:50.211629 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe3855-2a41-4dbe-a13c-9c1118d56223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:50.220280 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe3855-2a41-4dbe-a13c-9c1118d56223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:50.221904 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbe3855-2a41-4dbe-a13c-9c1118d56223 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:50.231367 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1359/2725] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:50.250298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbea93-151a-4f8a-9785-4c433ffa7b59 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:50.310936 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbea93-151a-4f8a-9785-4c433ffa7b59 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:50.315720 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:50.316617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:50.316896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:50.317168 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] PATH_INFO: `/v3/users/ceac655fafcd4843abf0a290b412dfe2` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:50.318387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceac655fafcd4843abf0a290b412dfe2)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:50.320450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:50.483349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceac655fafcd4843abf0a290b412dfe2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:50.490082 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceac655fafcd4843abf0a290b412dfe2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:50.490761 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceac655fafcd4843abf0a290b412dfe2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:50.503145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceac655fafcd4843abf0a290b412dfe2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:50.555507 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceac655fafcd4843abf0a290b412dfe2'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:50.562972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1367/2726] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:48:50 2022] DELETE /identity/v3/users/ceac655fafcd4843abf0a290b412dfe2 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:48:50.567497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:50.567785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:50.568058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:50.585724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:50.587206 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:50.654902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1360/2727] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:50.664598 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:50.666641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:50.700288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:50.722237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194c074e-d52f-48f8-9643-3215089671d0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:50.725901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683f09d-dab2-4531-9512-c2cb44d5778d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:50.726542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f09d-dab2-4531-9512-c2cb44d5778d service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:50.726826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f09d-dab2-4531-9512-c2cb44d5778d service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:50.727144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683f09d-dab2-4531-9512-c2cb44d5778d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:50.737186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683f09d-dab2-4531-9512-c2cb44d5778d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:50.738593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683f09d-dab2-4531-9512-c2cb44d5778d service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:50.748908 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1368/2728] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:51.088248 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:51.089791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:51.105002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff920c-323d-4740-9d6a-e09126aeb0f4 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:51.110076 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:51.110995 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.111258 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.111539 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] PATH_INFO: `/v3/projects/691b58b096fa43c49da6eaf9a6e7de36` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.112771 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=691b58b096fa43c49da6eaf9a6e7de36)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:51.114696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:51.195834 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:51.207804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '691b58b096fa43c49da6eaf9a6e7de36', 'request_id': 'req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:51.292466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '691b58b096fa43c49da6eaf9a6e7de36', 'request_id': 'req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:51.299111 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '691b58b096fa43c49da6eaf9a6e7de36', 'request_id': 'req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:51.311072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] The token cache is being invalidate because project 691b58b096fa43c49da6eaf9a6e7de36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:48:51.311357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:51.311979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e9e5bb18-5b75-47ef-8be7-bce1cfef5ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:48:51.313547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1361/2729] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:48:51 2022] DELETE /identity/v3/projects/691b58b096fa43c49da6eaf9a6e7de36 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:51.318536 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde6d4e-72b9-4680-a2eb-87f39aecddd8 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.318824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde6d4e-72b9-4680-a2eb-87f39aecddd8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.319103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde6d4e-72b9-4680-a2eb-87f39aecddd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.336399 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4dde6d4e-72b9-4680-a2eb-87f39aecddd8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:51.337915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4dde6d4e-72b9-4680-a2eb-87f39aecddd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:51.396137 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1369/2730] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:51.403935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd90863-82b3-49a8-aa87-d0efea47c0f5 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.404226 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd90863-82b3-49a8-aa87-d0efea47c0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.404505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd90863-82b3-49a8-aa87-d0efea47c0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.423113 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-2dd90863-82b3-49a8-aa87-d0efea47c0f5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:51.424294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-2dd90863-82b3-49a8-aa87-d0efea47c0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:51.432571 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1362/2731] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:51.436591 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde6d4e-72b9-4680-a2eb-87f39aecddd8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:51.446062 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde6d4e-72b9-4680-a2eb-87f39aecddd8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:51.450693 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:51.451338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.451619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.451895 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.452617 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:48:51.452919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:51.454544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:51.466212 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1370/2732] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:48:51.470281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd90863-82b3-49a8-aa87-d0efea47c0f5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:51.480102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd90863-82b3-49a8-aa87-d0efea47c0f5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:51.484704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:51.485375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.485658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.485940 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.486627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:51.488491 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:51.586532 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1363/2733] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:51 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:51.596572 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:51.602079 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e26da67-6b3d-467d-9871-239f20bf06bc admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:51.661634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:51.662312 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.662594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.662871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.663676 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:51.666339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:51.756490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1371/2734] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:51 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:48:51.759984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:51.770299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265c143-eef9-4d92-8117-61f68d3c523c admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:51.775754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:51.776406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.776688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.777011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.777861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:51.779583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:51.789718 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1364/2735] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:51.795147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:51.805151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7d3f01-c498-4d90-9741-93e18e32ae51 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:51.809817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4cbac5-bc01-45e5-8d2b-2fe9a9be9933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:51.810474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cbac5-bc01-45e5-8d2b-2fe9a9be9933 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.810762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cbac5-bc01-45e5-8d2b-2fe9a9be9933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.811046 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4cbac5-bc01-45e5-8d2b-2fe9a9be9933 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.811834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cbac5-bc01-45e5-8d2b-2fe9a9be9933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:51.813466 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4cbac5-bc01-45e5-8d2b-2fe9a9be9933 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:51.827189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1372/2736] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:51.828322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:51.840584 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c32fda-6a8d-4fb1-bff4-6638d50a465e admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:51.845520 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:51.846448 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.846728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.847002 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] PATH_INFO: `/v3/projects/c54481b2b5884d7f8c6901ab3f3eeefa/users/b0c2c28d91cf421ca6c336a1b4cddc93/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.862505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c54481b2b5884d7f8c6901ab3f3eeefa, user_id=b0c2c28d91cf421ca6c336a1b4cddc93, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:51.865155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:51.927279 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1365/2737] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:51 2022] PUT /identity/v3/projects/c54481b2b5884d7f8c6901ab3f3eeefa/users/b0c2c28d91cf421ca6c336a1b4cddc93/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:48:51.938053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7662a-5298-4dea-9ff8-d5fb72bb7193 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:51.938341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7662a-5298-4dea-9ff8-d5fb72bb7193 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:51.938618 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7662a-5298-4dea-9ff8-d5fb72bb7193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:51.966042 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-69a7662a-5298-4dea-9ff8-d5fb72bb7193 None None] MFA Rules not processed for user `b0c2c28d91cf421ca6c336a1b4cddc93`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:51.967231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-69a7662a-5298-4dea-9ff8-d5fb72bb7193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:51.996640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:52.032511 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1373/2738] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:52.036997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:52.112550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69928276-7ab2-48b4-a1f1-9ac1030238d0 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:52.121117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:52.121836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.122119 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.122393 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.123092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:52.125038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:52.215175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1366/2739] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:52 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:48:52.219181 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7662a-5298-4dea-9ff8-d5fb72bb7193 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:52.231006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7662a-5298-4dea-9ff8-d5fb72bb7193 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:52.289383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:52.290052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.290354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.291260 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.292111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:52.294061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:52.418997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1374/2740] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:52 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:48:52.423140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:52.433533 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee591621-abb0-491d-8ae2-1eddef7dd726 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:52.438941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:52.439693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.439998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.440281 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.441040 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:52.442684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:52.453421 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1367/2741] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:52.457545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:52.468574 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048fbac0-db8b-483d-bc4c-1b30dc04bca7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:52.473222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5fd0e-4ac4-40eb-b216-821864ffef8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:52.473905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5fd0e-4ac4-40eb-b216-821864ffef8a admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.474186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5fd0e-4ac4-40eb-b216-821864ffef8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.474457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5fd0e-4ac4-40eb-b216-821864ffef8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.475172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5fd0e-4ac4-40eb-b216-821864ffef8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:52.476786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5fd0e-4ac4-40eb-b216-821864ffef8a admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:52.487500 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1375/2742] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:52.491601 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:52.501725 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bff7c-19c2-4ef7-985e-90d3fde60954 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:52.506226 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:52.507098 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.507383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.507660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] PATH_INFO: `/v3/projects/9a00d64e4555498b8f16036fc2b24681/users/e7914f7c22704615bff9ac755a52af7d/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.522144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a00d64e4555498b8f16036fc2b24681, user_id=e7914f7c22704615bff9ac755a52af7d, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:52.525260 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:52.590155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1368/2743] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:52 2022] PUT /identity/v3/projects/9a00d64e4555498b8f16036fc2b24681/users/e7914f7c22704615bff9ac755a52af7d/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:48:52.601394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b55b0-6090-4e39-bb74-fb226a271936 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.601894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b55b0-6090-4e39-bb74-fb226a271936 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.602037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b55b0-6090-4e39-bb74-fb226a271936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.628378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fc8b55b0-6090-4e39-bb74-fb226a271936 None None] MFA Rules not processed for user `e7914f7c22704615bff9ac755a52af7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:52.629604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fc8b55b0-6090-4e39-bb74-fb226a271936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:52.692087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1376/2744] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:52.697302 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:52.711336 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:52.772257 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824bf8d4-4468-496a-bff7-4e8297bc1c84 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:52.777927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:52.778595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.778886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.779165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.779914 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:52.781916 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:52.888237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1369/2745] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:48:52 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:52.892586 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b55b0-6090-4e39-bb74-fb226a271936 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:52.903893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b55b0-6090-4e39-bb74-fb226a271936 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:52.936249 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:52.958864 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:52.959485 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:52.959751 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:52.960030 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:52.960727 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:52.963836 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:53.035305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1377/2746] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:48:52 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:48:53.039153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:53.049954 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72123ac7-ed38-4cb6-9952-8fb6128f2021 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:53.054651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:53.055309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:53.055595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:53.056392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:53.057453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:53.059053 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:53.069617 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1370/2747] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:48:53.074478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:53.084453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2219a84-caf1-4a9b-a3a8-7d87bf117b75 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:53.089739 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:53.090632 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:53.090930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:53.091207 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] PATH_INFO: `/v3/projects/5c3aab97db724c4e8e3f3203ef2ec119/users/2b1d4a4716314dcf94577966cf58d34e/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:53.106268 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3aab97db724c4e8e3f3203ef2ec119, user_id=2b1d4a4716314dcf94577966cf58d34e, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:53.108906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:53.224634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1378/2748] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:53 2022] PUT /identity/v3/projects/5c3aab97db724c4e8e3f3203ef2ec119/users/2b1d4a4716314dcf94577966cf58d34e/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:48:53.228671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:53.238825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ca86a-d2ad-4422-8916-c176de99fa59 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:53.297424 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c9ca0-5ce5-49f2-9dbf-b964b72941ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:53.298092 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c9ca0-5ce5-49f2-9dbf-b964b72941ac admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:53.298377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c9ca0-5ce5-49f2-9dbf-b964b72941ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:53.298654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c9ca0-5ce5-49f2-9dbf-b964b72941ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:53.299375 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c9ca0-5ce5-49f2-9dbf-b964b72941ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:53.300974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c9ca0-5ce5-49f2-9dbf-b964b72941ac admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:53.311176 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1371/2749] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:48:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:48:53.315416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:53.326906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f806d0-9c66-4513-858e-5c8701c5e3b5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:53.331626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:53.332512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:53.332796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:53.333072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] PATH_INFO: `/v3/projects/5c3aab97db724c4e8e3f3203ef2ec119/users/2b1d4a4716314dcf94577966cf58d34e/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:53.335061 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3aab97db724c4e8e3f3203ef2ec119, user_id=2b1d4a4716314dcf94577966cf58d34e, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:53.338340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:53.348645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1379/2750] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:48:53 2022] PUT /identity/v3/projects/5c3aab97db724c4e8e3f3203ef2ec119/users/2b1d4a4716314dcf94577966cf58d34e/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:48:53.359193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5bab-db4d-4c8f-8112-55e76b02b87e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:53.359507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5bab-db4d-4c8f-8112-55e76b02b87e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:53.359789 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca5bab-db4d-4c8f-8112-55e76b02b87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:53.388630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-fdca5bab-db4d-4c8f-8112-55e76b02b87e None None] MFA Rules not processed for user `2b1d4a4716314dcf94577966cf58d34e`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:48:53.389866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-fdca5bab-db4d-4c8f-8112-55e76b02b87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:53.462852 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1372/2751] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:48:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:48:53.478662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:53.480641 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:48:53.537282 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d5086-50aa-4995-89a9-9ec8d17d5557 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:53.541526 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:53.542173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:53.542453 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:53.542728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:53.598167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:53.599607 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:53.608956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1380/2752] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:55.536241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5bab-db4d-4c8f-8112-55e76b02b87e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:55.541191 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:48:55.546920 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca5bab-db4d-4c8f-8112-55e76b02b87e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:55.550681 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:55.551322 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:55.551607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:55.551882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:55.611159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:55.612603 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:55.622161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1373/2753] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:48:57.068147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:48:57.077757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0b741-85c1-4ee5-8526-5be12019d733 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:48:57.081704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15407e-436c-438d-9767-35d981acd100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:48:57.082631 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15407e-436c-438d-9767-35d981acd100 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:48:57.082754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15407e-436c-438d-9767-35d981acd100 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:48:57.083018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15407e-436c-438d-9767-35d981acd100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:48:57.092688 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15407e-436c-438d-9767-35d981acd100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:48:57.094146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15407e-436c-438d-9767-35d981acd100 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:48:57.103432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1381/2754] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:48:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:03.857655 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:03.919509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828a6a6-b11a-498c-9961-a67d4976e27c service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:03.923987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:03.924853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:03.925136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:03.925451 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] PATH_INFO: `/v3/users/b0c2c28d91cf421ca6c336a1b4cddc93` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:03.926610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0c2c28d91cf421ca6c336a1b4cddc93)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:03.928507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:04.113453 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0c2c28d91cf421ca6c336a1b4cddc93'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:04.114037 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0c2c28d91cf421ca6c336a1b4cddc93'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:04.121685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0c2c28d91cf421ca6c336a1b4cddc93'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:04.122305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0c2c28d91cf421ca6c336a1b4cddc93'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:04.133951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0c2c28d91cf421ca6c336a1b4cddc93'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:04.212414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1374/2755] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:49:03 2022] DELETE /identity/v3/users/b0c2c28d91cf421ca6c336a1b4cddc93 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:04.217072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:04.217587 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:04.217694 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:04.245546 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:04.246734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:04.262597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:04.308492 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1382/2756] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:49:04.320779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:04.322199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:04.345467 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:04.381370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffd66c-79e1-4e08-8028-713e7e8fbf6b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:04.384952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:04.385654 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:04.386010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:04.386315 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:04.395079 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:04.396509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:04.406477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1375/2757] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:04.802682 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:04.804153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:04.817301 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e48a2-4890-4a69-8d31-17bc83d66f7b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:04.821997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:04.823067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:04.823205 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:04.823442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] PATH_INFO: `/v3/projects/c54481b2b5884d7f8c6901ab3f3eeefa` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:04.824743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c54481b2b5884d7f8c6901ab3f3eeefa)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:04.826717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:04.949747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c54481b2b5884d7f8c6901ab3f3eeefa', 'request_id': 'req-176eeb93-6c35-4cb5-8fcb-e8972909fe11'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.017209 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c54481b2b5884d7f8c6901ab3f3eeefa', 'request_id': 'req-176eeb93-6c35-4cb5-8fcb-e8972909fe11'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.024193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c54481b2b5884d7f8c6901ab3f3eeefa', 'request_id': 'req-176eeb93-6c35-4cb5-8fcb-e8972909fe11'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.035050 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] The token cache is being invalidate because project c54481b2b5884d7f8c6901ab3f3eeefa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:49:05.035335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.035951 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.037655 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1383/2758] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:49:04 2022] DELETE /identity/v3/projects/c54481b2b5884d7f8c6901ab3f3eeefa => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:05.042511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:05.052378 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bce4cb-8b34-4ef2-a595-3511a0ce8e38 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:05.110670 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:05.111554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:05.111843 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:05.112121 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] PATH_INFO: `/v3/users/e7914f7c22704615bff9ac755a52af7d` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:05.113442 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7914f7c22704615bff9ac755a52af7d)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:05.115292 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:05.247836 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7914f7c22704615bff9ac755a52af7d'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.248492 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7914f7c22704615bff9ac755a52af7d'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.255685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7914f7c22704615bff9ac755a52af7d'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.256363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7914f7c22704615bff9ac755a52af7d'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.268235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7914f7c22704615bff9ac755a52af7d'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:05.326141 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1376/2759] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:49:05 2022] DELETE /identity/v3/users/e7914f7c22704615bff9ac755a52af7d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:05.722642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:05.724141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:05.738942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eeb93-6c35-4cb5-8fcb-e8972909fe11 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:05.797522 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:05.798393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:05.798681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:05.798961 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] PATH_INFO: `/v3/projects/9a00d64e4555498b8f16036fc2b24681` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:05.800134 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a00d64e4555498b8f16036fc2b24681)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:05.802089 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:05.931560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:05.933760 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a00d64e4555498b8f16036fc2b24681', 'request_id': 'req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.000112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a00d64e4555498b8f16036fc2b24681', 'request_id': 'req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.006654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a00d64e4555498b8f16036fc2b24681', 'request_id': 'req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.017950 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] The token cache is being invalidate because project 9a00d64e4555498b8f16036fc2b24681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:49:06.018237 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.018865 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.020398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1384/2760] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:49:05 2022] DELETE /identity/v3/projects/9a00d64e4555498b8f16036fc2b24681 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:06.024422 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:06.025882 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:06.038708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba4892e-e3ec-4e34-bfdc-232b742d65c8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:06.080132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:06.096641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:06.097547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:06.097830 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:06.098104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] PATH_INFO: `/v3/users/2b1d4a4716314dcf94577966cf58d34e` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:06.099299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1d4a4716314dcf94577966cf58d34e)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:06.101230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:06.272389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1d4a4716314dcf94577966cf58d34e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.273052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1d4a4716314dcf94577966cf58d34e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.281369 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1d4a4716314dcf94577966cf58d34e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.282036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1d4a4716314dcf94577966cf58d34e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.295427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1d4a4716314dcf94577966cf58d34e'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:06.373832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1377/2761] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:49:06 2022] DELETE /identity/v3/users/2b1d4a4716314dcf94577966cf58d34e => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:06.819670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:06.821035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:06.840400 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcf310-2ee7-4f0b-8ec0-a5163b18de38 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:06.905475 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:06.906373 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:06.906654 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:06.906930 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] PATH_INFO: `/v3/projects/5c3aab97db724c4e8e3f3203ef2ec119` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:06.908112 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c3aab97db724c4e8e3f3203ef2ec119)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:06.910372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:07.052339 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c3aab97db724c4e8e3f3203ef2ec119', 'request_id': 'req-20416ad6-438d-43be-8d71-b7f3cf801e1d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:07.114831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c3aab97db724c4e8e3f3203ef2ec119', 'request_id': 'req-20416ad6-438d-43be-8d71-b7f3cf801e1d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:07.121729 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c3aab97db724c4e8e3f3203ef2ec119', 'request_id': 'req-20416ad6-438d-43be-8d71-b7f3cf801e1d'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:07.132064 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] The token cache is being invalidate because project 5c3aab97db724c4e8e3f3203ef2ec119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:49:07.132348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:07.132978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:07.134545 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1385/2762] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:49:06 2022] DELETE /identity/v3/projects/5c3aab97db724c4e8e3f3203ef2ec119 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:11.138502 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:11.139211 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:11.195786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d78ed4-ed72-40f1-82e4-01a87432fc8f admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:11.244152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:11.245490 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:11.245762 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:11.246034 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:11.247812 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:11.277497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:11.307608 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:11.309536 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:11.319323 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1378/2763] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:11.811374 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:11.815197 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:11.885199 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20416ad6-438d-43be-8d71-b7f3cf801e1d admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:11.890510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:11.891143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:11.891429 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:11.892160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:11.893594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:11.903540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:11.906039 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:11.907733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:11.918723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1386/2764] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:11.932476 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:11.933497 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:11.947010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c8694-6cea-4560-9add-2755757a8d15 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:11.953259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:11.954277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:11.954565 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:11.954844 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] PATH_INFO: `/v3/users/bb2d87f281834e4697a8c3559b6e83a2` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:11.956103 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb2d87f281834e4697a8c3559b6e83a2)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:11.958070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:12.140189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb2d87f281834e4697a8c3559b6e83a2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.141184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb2d87f281834e4697a8c3559b6e83a2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.148878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb2d87f281834e4697a8c3559b6e83a2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.149548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb2d87f281834e4697a8c3559b6e83a2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.162136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-2cb30dec-4cfa-42c6-be29-2f96a3d7ad18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb2d87f281834e4697a8c3559b6e83a2'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.234501 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1379/2765] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:49:11 2022] DELETE /identity/v3/users/bb2d87f281834e4697a8c3559b6e83a2 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:49:12.498947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:12.513145 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496faf-7871-4abe-878a-cd9a0122be98 service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:12.572223 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:12.574035 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:12.574330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:12.574612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] PATH_INFO: `/v3/projects/0f0441599dc5456da3e0f48be7dd2d20` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:12.575893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0441599dc5456da3e0f48be7dd2d20)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:12.577879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:12.681081 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0441599dc5456da3e0f48be7dd2d20', 'request_id': 'req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.759886 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0441599dc5456da3e0f48be7dd2d20', 'request_id': 'req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.767065 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0441599dc5456da3e0f48be7dd2d20', 'request_id': 'req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.777589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] The token cache is being invalidate because project 0f0441599dc5456da3e0f48be7dd2d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:49:12.777878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.778513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-0f7ce14d-c2e5-4ec8-a8b4-fb9dc25e2870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:12.780063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1387/2766] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:49:12 2022] DELETE /identity/v3/projects/0f0441599dc5456da3e0f48be7dd2d20 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:12.785706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cd55e-6d7a-4a35-95a3-8033bb984615 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:12.786017 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cd55e-6d7a-4a35-95a3-8033bb984615 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:12.786294 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36cd55e-6d7a-4a35-95a3-8033bb984615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:12.804567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d36cd55e-6d7a-4a35-95a3-8033bb984615 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:12.805792 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d36cd55e-6d7a-4a35-95a3-8033bb984615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:12.864696 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1380/2767] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:12.873147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1434c8-f720-4755-b739-e2627567b5b8 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:12.873701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1434c8-f720-4755-b739-e2627567b5b8 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:12.873811 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1434c8-f720-4755-b739-e2627567b5b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:12.891681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-7b1434c8-f720-4755-b739-e2627567b5b8 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:12.892871 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7b1434c8-f720-4755-b739-e2627567b5b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:12.900683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1388/2768] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:49:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:12.905072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cd55e-6d7a-4a35-95a3-8033bb984615 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:12.914699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36cd55e-6d7a-4a35-95a3-8033bb984615 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:12.919946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:12.920590 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:12.920868 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:12.921146 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:12.921897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80404) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:49:12.922203 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:12.923786 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:12.935820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1381/2769] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:49:12.939890 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1434c8-f720-4755-b739-e2627567b5b8 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:12.949163 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1434c8-f720-4755-b739-e2627567b5b8 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:12.953795 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:12.954436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:12.954718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:12.954994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:12.955699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:12.957614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:13.030497 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1389/2770] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:49:12 2022] POST /identity/v3/projects => generated 399 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:49:13.034445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:13.046358 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a8bd2-f8eb-43f8-aec4-a08c255a4f34 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:13.071014 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:13.104445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:13.105117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:13.105431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:13.105726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] PATH_INFO: `/v3/users` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:13.106429 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:13.108318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:13.206161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1382/2771] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:49:13 2022] POST /identity/v3/users => generated 300 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:49:13.210360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:13.221043 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b730ef6-71d5-42c8-9ec3-6ca59f0f3452 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:13.225762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:13.226419 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:13.226697 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:13.226969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:13.227722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:13.230123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:13.240530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1390/2772] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:49:13.245171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:13.255162 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b010ea3-11c0-4256-8ba6-509a053adc87 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:13.259679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f7a2e-8f22-4ec4-b7bd-7effa043d8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:13.260522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f7a2e-8f22-4ec4-b7bd-7effa043d8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:13.260807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f7a2e-8f22-4ec4-b7bd-7effa043d8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:13.261084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f7a2e-8f22-4ec4-b7bd-7effa043d8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:13.261840 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f7a2e-8f22-4ec4-b7bd-7effa043d8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:13.263423 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f7a2e-8f22-4ec4-b7bd-7effa043d8b2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:13.273001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1383/2773] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:49:13.277612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:13.287608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d184bef8-8b37-4bc7-9310-b80fed877fb0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:13.292805 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:13.293721 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:13.294005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:13.294285 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] PATH_INFO: `/v3/projects/7125c738819942989315f6e28cf3707c/users/bcb0a97fae4e4f64bfa0da8c152b6945/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:13.309914 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7125c738819942989315f6e28cf3707c, user_id=bcb0a97fae4e4f64bfa0da8c152b6945, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:13.312603 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:13.383071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1391/2774] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:49:13 2022] PUT /identity/v3/projects/7125c738819942989315f6e28cf3707c/users/bcb0a97fae4e4f64bfa0da8c152b6945/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:13.392951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ac5ae-6247-4f62-8277-60d07cc757b7 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:13.393239 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ac5ae-6247-4f62-8277-60d07cc757b7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:13.393546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74ac5ae-6247-4f62-8277-60d07cc757b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:13.421151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d74ac5ae-6247-4f62-8277-60d07cc757b7 None None] MFA Rules not processed for user `bcb0a97fae4e4f64bfa0da8c152b6945`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:13.422390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d74ac5ae-6247-4f62-8277-60d07cc757b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:13.487982 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1384/2775] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:49:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:49:13.499144 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:13.500796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:13.527077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:13.567175 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295593d-eed7-4094-9da2-f74310cce17f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:13.570983 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edea951-0308-4d7d-98fa-8f8912477738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:13.573173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edea951-0308-4d7d-98fa-8f8912477738 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:13.573173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edea951-0308-4d7d-98fa-8f8912477738 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:13.573173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edea951-0308-4d7d-98fa-8f8912477738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:13.581022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edea951-0308-4d7d-98fa-8f8912477738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:13.582552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edea951-0308-4d7d-98fa-8f8912477738 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:13.593009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1392/2776] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:22.342943 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ac5ae-6247-4f62-8277-60d07cc757b7 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:22.403738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74ac5ae-6247-4f62-8277-60d07cc757b7 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:22.408082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:22.408945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:22.409229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:22.409540 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] PATH_INFO: `/v3/users/bcb0a97fae4e4f64bfa0da8c152b6945` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:22.410698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb0a97fae4e4f64bfa0da8c152b6945)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:22.412592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:22.548693 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:22.557109 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb0a97fae4e4f64bfa0da8c152b6945'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:22.557839 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb0a97fae4e4f64bfa0da8c152b6945'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:22.564598 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb0a97fae4e4f64bfa0da8c152b6945'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:22.565238 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb0a97fae4e4f64bfa0da8c152b6945'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:22.577067 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb0a97fae4e4f64bfa0da8c152b6945'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:22.679669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1385/2777] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:49:22 2022] DELETE /identity/v3/users/bcb0a97fae4e4f64bfa0da8c152b6945 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:22.684380 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:22.690906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:22.690906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:22.702831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:22.704027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:22.762308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1393/2778] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:49:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:22.771886 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:22.773290 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:22.830659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ffe31-f472-458d-97c1-15297f7d0698 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:22.834259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee051e2-da7a-4608-ab99-0681646600ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:22.834902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee051e2-da7a-4608-ab99-0681646600ec service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:22.835186 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee051e2-da7a-4608-ab99-0681646600ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:22.835464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee051e2-da7a-4608-ab99-0681646600ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:22.845374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee051e2-da7a-4608-ab99-0681646600ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:22.846801 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee051e2-da7a-4608-ab99-0681646600ec service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:22.856316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1386/2779] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:23.247696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:23.256190 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:23.262184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39f547f-6baa-46f6-9638-9b3eb7d5ebbc None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:23.267151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:23.268069 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:23.268351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:23.268628 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] PATH_INFO: `/v3/projects/7125c738819942989315f6e28cf3707c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:23.269847 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7125c738819942989315f6e28cf3707c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:23.271902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:23.393862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7125c738819942989315f6e28cf3707c', 'request_id': 'req-7ced31d1-1544-49dc-857a-dad10acab78f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:23.491353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7125c738819942989315f6e28cf3707c', 'request_id': 'req-7ced31d1-1544-49dc-857a-dad10acab78f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:23.498746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7125c738819942989315f6e28cf3707c', 'request_id': 'req-7ced31d1-1544-49dc-857a-dad10acab78f'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:23.509251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] The token cache is being invalidate because project 7125c738819942989315f6e28cf3707c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:49:23.509624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:23.510369 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:49:23.512211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1394/2780] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:49:23 2022] DELETE /identity/v3/projects/7125c738819942989315f6e28cf3707c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:49:32.851965 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=41,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:33.506167 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:36.897516 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5128553-8dac-424f-a9c3-772097faca8f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:36.898010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5128553-8dac-424f-a9c3-772097faca8f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:36.898144 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5128553-8dac-424f-a9c3-772097faca8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:36.956122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b5128553-8dac-424f-a9c3-772097faca8f None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:36.957385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b5128553-8dac-424f-a9c3-772097faca8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:37.028152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1387/2781] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:49:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:37.191882 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:37.193384 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:37.285662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced31d1-1544-49dc-857a-dad10acab78f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:37.290624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff46f97b-d880-4ecc-b500-0577cb2bfdcf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:37.291275 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46f97b-d880-4ecc-b500-0577cb2bfdcf service swift] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:37.291579 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46f97b-d880-4ecc-b500-0577cb2bfdcf service swift] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:37.291849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff46f97b-d880-4ecc-b500-0577cb2bfdcf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:37.300831 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46f97b-d880-4ecc-b500-0577cb2bfdcf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:37.302428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff46f97b-d880-4ecc-b500-0577cb2bfdcf service swift] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:37.310874 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1395/2782] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:49:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:49:47.024487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=47 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:47.307386 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=41 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:49:53.630503 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462197e-be9c-4bc6-bece-a1cdcf1f963a None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:53.631101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462197e-be9c-4bc6-bece-a1cdcf1f963a None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:53.631218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462197e-be9c-4bc6-bece-a1cdcf1f963a None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:53.632575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1388/2783] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:49:53.638513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6aa073-7c7b-472e-9e4e-5e852289a3e7 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:53.639583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6aa073-7c7b-472e-9e4e-5e852289a3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:53.640048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6aa073-7c7b-472e-9e4e-5e852289a3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:53.689329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6c6aa073-7c7b-472e-9e4e-5e852289a3e7 None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:53.690573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6c6aa073-7c7b-472e-9e4e-5e852289a3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:53.765754 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1396/2784] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:49:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:53.777147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462197e-be9c-4bc6-bece-a1cdcf1f963a None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:53.778823 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-c462197e-be9c-4bc6-bece-a1cdcf1f963a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:53.836756 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462197e-be9c-4bc6-bece-a1cdcf1f963a None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:53.840317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a3a5-2202-4900-846b-c792a4ec8a28 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9af9eae0fa5b437e96c1c732dfe4fa7a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:53.840962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a3a5-2202-4900-846b-c792a4ec8a28 service glance] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:53.841249 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a3a5-2202-4900-846b-c792a4ec8a28 service glance] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:53.841573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a3a5-2202-4900-846b-c792a4ec8a28 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:53.851567 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9a3a5-2202-4900-846b-c792a4ec8a28 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:53.853001 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9a3a5-2202-4900-846b-c792a4ec8a28 service glance] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:53.863742 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1389/2785] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:54.046168 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a822c5d-97e2-4bf7-b3a1-f651470a3c56 None None] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:54.046383 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a822c5d-97e2-4bf7-b3a1-f651470a3c56 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:54.046645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a822c5d-97e2-4bf7-b3a1-f651470a3c56 None None] PATH_INFO: `/` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:54.048058 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1397/2786] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:49:54.052795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7429f-f72c-4830-8b21-ba8da7ac6095 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:54.053084 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7429f-f72c-4830-8b21-ba8da7ac6095 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:54.053390 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b7429f-f72c-4830-8b21-ba8da7ac6095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:54.089100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b1b7429f-f72c-4830-8b21-ba8da7ac6095 None None] MFA Rules not processed for user `9937f1088b304574935fcdf579e4de7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:54.090318 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b1b7429f-f72c-4830-8b21-ba8da7ac6095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:54.141235 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1390/2787] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:49:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:54.152905 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a822c5d-97e2-4bf7-b3a1-f651470a3c56 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:54.154640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6a822c5d-97e2-4bf7-b3a1-f651470a3c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:54.168101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a822c5d-97e2-4bf7-b3a1-f651470a3c56 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:54.172284 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c7546-a70a-42ed-b1e7-5f59907d77dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:54.172924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c7546-a70a-42ed-b1e7-5f59907d77dd service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:54.173208 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c7546-a70a-42ed-b1e7-5f59907d77dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:54.173518 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c7546-a70a-42ed-b1e7-5f59907d77dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:54.180639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c7546-a70a-42ed-b1e7-5f59907d77dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:54.182101 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c7546-a70a-42ed-b1e7-5f59907d77dd service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:54.190342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1398/2788] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:49:54.277507 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1d9a-bcd9-43e9-83db-85b31c8866be None None] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:54.277800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1d9a-bcd9-43e9-83db-85b31c8866be None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:54.278081 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c1d9a-bcd9-43e9-83db-85b31c8866be None None] PATH_INFO: `/` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:54.279505 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1391/2789] 10.222.0.35 () {54 vars in 921 bytes} [Tue May 24 10:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 24 10:49:54.284298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da125c53-4b7b-415c-b801-b303df9e2f74 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:54.284584 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da125c53-4b7b-415c-b801-b303df9e2f74 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:54.284885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da125c53-4b7b-415c-b801-b303df9e2f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:54.303768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-da125c53-4b7b-415c-b801-b303df9e2f74 None None] MFA Rules not processed for user `8de75a1df6784111aabec60da40ab756`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:54.304998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-da125c53-4b7b-415c-b801-b303df9e2f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:54.315666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1399/2790] 10.222.0.35 () {58 vars in 1018 bytes} [Tue May 24 10:49:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:54.325885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1d9a-bcd9-43e9-83db-85b31c8866be None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:54.337348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c1d9a-bcd9-43e9-83db-85b31c8866be None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:54.340951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:54.344099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:54.344099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:54.344099 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:54.353867 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:54.355293 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:54.366798 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1392/2791] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:49:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:49:57.856844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae094e-f316-4682-83c3-14eb3395fe9e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:57.857393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae094e-f316-4682-83c3-14eb3395fe9e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:57.857393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae094e-f316-4682-83c3-14eb3395fe9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:57.876417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-daae094e-f316-4682-83c3-14eb3395fe9e None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:49:57.877678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-daae094e-f316-4682-83c3-14eb3395fe9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:49:57.885044 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1400/2792] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:49:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:49:57.897796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:49:57.909574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bc929-2c14-4975-8205-50db303eef01 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:49:57.914934 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:49:57.915671 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:49:57.916028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:49:57.916371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:49:57.925123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:49:57.926807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:49:57.935131 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1393/2793] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:49:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:50:07.870381 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=58 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:50:07.932171 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=67 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:50:22.283341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd276de-59a9-4bf1-86c2-909bd3e8e521 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:50:22.283835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd276de-59a9-4bf1-86c2-909bd3e8e521 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:50:22.283945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd276de-59a9-4bf1-86c2-909bd3e8e521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:50:22.301780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fbd276de-59a9-4bf1-86c2-909bd3e8e521 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:50:22.302967 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fbd276de-59a9-4bf1-86c2-909bd3e8e521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:50:22.310045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1401/2794] 10.222.0.35 () {58 vars in 967 bytes} [Tue May 24 10:50:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:50:22.322153 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:50:22.332548 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c943ab-8b6c-4463-bb5b-cd5b9f45787e service swift] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:50:22.337842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:50:22.338499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:50:22.338782 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:50:22.339058 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:50:22.347419 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:50:22.348980 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:50:22.357009 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1394/2795] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:50:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 10:50:32.295948 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:50:32.353880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:51:07.663717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:51:07.666469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:51:07.666469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:51:07.682349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] MFA Rules not processed for user `0e46c23c8e3946238795b800b921380c`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:51:07.683612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:51:07.738253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1402/2796] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:51:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:51:17.731307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=25 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:51:59.852699 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:51:59.872580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182da3f-72a8-4aee-870c-834d0cc5286e service swift] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:51:59.876157 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed7107-9d29-4434-bf42-e4bc0466a4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:51:59.876790 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7107-9d29-4434-bf42-e4bc0466a4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:51:59.877073 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7107-9d29-4434-bf42-e4bc0466a4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:51:59.877377 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed7107-9d29-4434-bf42-e4bc0466a4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:51:59.885804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-80ed7107-9d29-4434-bf42-e4bc0466a4f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:51:59.901893 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7107-9d29-4434-bf42-e4bc0466a4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:51:59.903324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ed7107-9d29-4434-bf42-e4bc0466a4f5 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:51:59.911750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1395/2797] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:51:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:52:04.143001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:52:04.144405 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:52:04.227896 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1bfb0-1ff9-4e8f-9782-8b314b22ebd7 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:52:04.233510 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:52:04.234106 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:52:04.234390 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:52:04.234666 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:52:04.236436 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:52:04.248279 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:52:04.249898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:52:04.259640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1403/2798] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:52:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:52:09.907876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=15 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:14.255432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=37 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:22.783988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339c79-ed73-4a1c-bd7d-6ad1f7d0aaf7 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:52:22.783988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339c79-ed73-4a1c-bd7d-6ad1f7d0aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:52:22.783988 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7339c79-ed73-4a1c-bd7d-6ad1f7d0aaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:52:22.803054 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a7339c79-ed73-4a1c-bd7d-6ad1f7d0aaf7 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:52:22.804462 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a7339c79-ed73-4a1c-bd7d-6ad1f7d0aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:52:22.812478 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1396/2799] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:52:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:52:22.942444 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:52:22.954960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b12ce-e21f-41e9-8882-9ff14aae0d36 service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:52:22.960317 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:52:22.960969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:52:22.961300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:52:22.961583 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:52:22.971011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:52:22.972594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:52:22.980629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1404/2800] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:52:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 24 10:52:32.797582 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:32.977494 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:41.554574 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1413c-e409-4a92-808a-2ba35adb16e1 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:52:41.555075 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1413c-e409-4a92-808a-2ba35adb16e1 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:52:41.555193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b1413c-e409-4a92-808a-2ba35adb16e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:52:41.573644 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e8b1413c-e409-4a92-808a-2ba35adb16e1 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:52:41.574855 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e8b1413c-e409-4a92-808a-2ba35adb16e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:52:41.582605 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1397/2801] 10.222.0.35 () {58 vars in 1009 bytes} [Tue May 24 10:52:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:52:41.595432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:52:41.606037 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a6a76-f09a-4c24-ad7f-8018d64e1497 service swift] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:52:41.610926 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:52:41.611560 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:52:41.611839 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:52:41.612110 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:52:41.620195 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:52:41.621797 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:52:41.629647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1405/2802] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:52:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:52:51.568200 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:51.626450 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:52:56.752406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:52:56.754889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:52:56.754889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:52:56.770651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] MFA Rules not processed for user `70ebdb4e82064aad89b0dcec0a74cad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:52:56.771845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:52:56.778962 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1398/2803] 10.222.0.35 () {58 vars in 967 bytes} [Tue May 24 10:52:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:52:56.791299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:52:56.802553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ce41d-f69c-45cf-bc50-1b2ad15b57b7 service swift] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:52:56.807539 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05f46d019e1348adabd023de481f0a3d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:52:56.808253 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:52:56.808685 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:52:56.808959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:52:56.816956 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:52:56.818558 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:52:56.827184 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1406/2804] 10.222.0.35 () {60 vars in 1384 bytes} [Tue May 24 10:52:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 24 10:53:06.765609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:06.823222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=6 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:31.807807 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:31.809218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:31.877242 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f0f05-2c8e-4b2e-9f15-f772223cc8f5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:31.881020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:31.881704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:31.881993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:31.882277 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:31.884065 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:31.895588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:31.897011 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:31.906769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1399/2805] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:53:32.399920 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:32.401428 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:32.413790 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231fece-fb67-4047-a1ad-11ea6faf2fc0 service swift] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:32.417477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:32.418114 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:32.418394 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:32.418668 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:32.421103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:32.434504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:32.436095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:32.447173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1407/2806] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:53:34.833523 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:34.835751 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:34.853147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe35e56-fb26-45b8-b753-66e5c822c5ec service cinder] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:34.860708 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:34.861706 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:34.862061 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:34.862430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:34.864324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:34.880889 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:34.882551 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:34.896152 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1400/2807] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:53:41.766593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:41.775276 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d5e7d-8744-4756-854b-267d62f2832d service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:41.778942 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:41.779578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:41.779862 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:41.780139 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:41.782512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:41.868504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:41.869952 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:41.889225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1408/2808] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:53:44.891184 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=37 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:45.281659 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:45.294250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f51591-9707-4109-948b-99e5f958e76a service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:45.298444 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:45.299115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:45.299436 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:45.299717 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:45.301711 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:45.314372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:45.315895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:45.326198 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1401/2809] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:53:45.403799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:45.405345 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:45.418056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26859d02-5e9e-4a82-bf87-ba97d10099ed service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:45.423103 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:45.423970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:45.424254 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:45.424531 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] PATH_INFO: `/v3/users/0e46c23c8e3946238795b800b921380c` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:45.425745 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e46c23c8e3946238795b800b921380c)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:45.427669 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:45.589998 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e46c23c8e3946238795b800b921380c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:45.597097 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e46c23c8e3946238795b800b921380c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:45.597799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e46c23c8e3946238795b800b921380c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:45.609846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e46c23c8e3946238795b800b921380c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:45.655019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-3f2c2372-4dad-44b4-82d4-23bec421c22e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e46c23c8e3946238795b800b921380c'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:45.661225 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1409/2810] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:53:45 2022] DELETE /identity/v3/users/0e46c23c8e3946238795b800b921380c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:53:45.902319 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:45.913667 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb5049-353c-48b1-992e-782dcbefe9c0 service placement] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:45.972361 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:45.973229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:45.973543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:45.973829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] PATH_INFO: `/v3/projects/107c9dab6bd24705b50f8d30181d6582` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:45.974998 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=107c9dab6bd24705b50f8d30181d6582)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:45.976892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:46.072140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107c9dab6bd24705b50f8d30181d6582', 'request_id': 'req-27b21e71-ea8d-4117-bccb-9064530117fa'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:46.129754 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107c9dab6bd24705b50f8d30181d6582', 'request_id': 'req-27b21e71-ea8d-4117-bccb-9064530117fa'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:46.135594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107c9dab6bd24705b50f8d30181d6582', 'request_id': 'req-27b21e71-ea8d-4117-bccb-9064530117fa'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:46.149379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] The token cache is being invalidate because project 107c9dab6bd24705b50f8d30181d6582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:53:46.149599 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:46.150240 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-27b21e71-ea8d-4117-bccb-9064530117fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:53:46.151858 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1402/2811] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:53:45 2022] DELETE /identity/v3/projects/107c9dab6bd24705b50f8d30181d6582 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:53:46.157425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7409610-7f7e-491c-a633-f758344f99ff None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.157715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7409610-7f7e-491c-a633-f758344f99ff None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.157994 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7409610-7f7e-491c-a633-f758344f99ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.191052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:46.216211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-e7409610-7f7e-491c-a633-f758344f99ff None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:53:46.217463 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-e7409610-7f7e-491c-a633-f758344f99ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:46.276495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1410/2812] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:53:46.284715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f2936-ace6-4f5a-8432-705ef56f2f73 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.285007 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f2936-ace6-4f5a-8432-705ef56f2f73 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.285314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f2936-ace6-4f5a-8432-705ef56f2f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.303041 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-e18f2936-ace6-4f5a-8432-705ef56f2f73 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:53:46.304263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-e18f2936-ace6-4f5a-8432-705ef56f2f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:46.312766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1403/2813] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:53:46.317005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7409610-7f7e-491c-a633-f758344f99ff None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:46.326624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7409610-7f7e-491c-a633-f758344f99ff None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:46.331828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:46.332733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.333016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.333353 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.334088 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:53:46.334397 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:46.335997 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:46.347528 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1411/2814] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:53:46.352360 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f2936-ace6-4f5a-8432-705ef56f2f73 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:46.361845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f2936-ace6-4f5a-8432-705ef56f2f73 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:46.366480 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:46.367134 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.367417 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.367695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.368379 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:46.370309 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:46.454403 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1404/2815] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:53:46 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:53:46.458964 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:46.469092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50d53b6-911f-4f3b-acb3-e0cfd5fc3959 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:46.529019 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:46.529722 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.530006 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.530281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.531086 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:46.533068 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:46.642495 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1412/2816] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:53:46 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:53:46.646521 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:46.656933 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d6e89-4db1-4a14-9173-5122b6e0f0cb admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:46.661441 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:46.662129 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.662411 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.662686 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.663402 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:46.664985 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:46.675879 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1405/2817] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:53:46.679872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:46.690367 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1accb-9145-4ff4-87d9-979f50727491 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:46.694945 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3db39-ce4a-4e3c-8453-91901b98fe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:46.695600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3db39-ce4a-4e3c-8453-91901b98fe5f admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.695879 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3db39-ce4a-4e3c-8453-91901b98fe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.696151 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3db39-ce4a-4e3c-8453-91901b98fe5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.696899 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3db39-ce4a-4e3c-8453-91901b98fe5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:46.698523 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3db39-ce4a-4e3c-8453-91901b98fe5f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:46.708704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1413/2818] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:53:46.712572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:46.722637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88808f8-f0af-4528-ad29-e60ee111f596 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:46.727161 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:46.728050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.728321 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.728602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] PATH_INFO: `/v3/projects/c94763dacd084cd69ee0729ee21d28a2/users/ea49ebe4f80d48399086cacfe7b4bfdb/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.743557 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c94763dacd084cd69ee0729ee21d28a2, user_id=ea49ebe4f80d48399086cacfe7b4bfdb, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:46.746291 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:46.829846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1406/2819] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:53:46 2022] PUT /identity/v3/projects/c94763dacd084cd69ee0729ee21d28a2/users/ea49ebe4f80d48399086cacfe7b4bfdb/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:53:46.835235 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d34b0-810d-4e8d-962f-b13443345d3e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.835629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d34b0-810d-4e8d-962f-b13443345d3e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.835989 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0d34b0-810d-4e8d-962f-b13443345d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.853594 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-9f0d34b0-810d-4e8d-962f-b13443345d3e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:53:46.854789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-9f0d34b0-810d-4e8d-962f-b13443345d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:46.903707 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:46.912243 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1414/2820] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 24 10:53:46.922522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:46.924263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:46.931122 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:53:46.981926 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49464f3-6a40-4ccf-b18c-4a30df7f8abf admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:46.986561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:46.987246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:46.987535 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:46.987818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:46.997118 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:46.998575 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:47.008863 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1407/2821] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:53:52.241170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb5761-40c4-4837-b146-a5d3698f8eb6 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:52.241737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb5761-40c4-4837-b146-a5d3698f8eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:52.242335 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb5761-40c4-4837-b146-a5d3698f8eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:52.271009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-2ddb5761-40c4-4837-b146-a5d3698f8eb6 None None] MFA Rules not processed for user `ea49ebe4f80d48399086cacfe7b4bfdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:53:52.272331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-2ddb5761-40c4-4837-b146-a5d3698f8eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:52.343659 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1415/2822] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:53:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:53:52.360615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:52.362089 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:52.442542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27843df2-a335-49b9-9073-0ac32ae4fd33 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:52.448108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:52.451255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:52.451255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:52.451255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:52.459027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:52.460614 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:52.471320 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1408/2823] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:53:54.680350 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:54.680872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:54.680872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:54.699146 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] MFA Rules not processed for user `ea49ebe4f80d48399086cacfe7b4bfdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:53:54.700357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:54.708468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1416/2824] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:53:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:53:55.031477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:55.033545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:55.046343 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e0e08-4664-4849-a358-d8c69e866bd7 service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:55.050426 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:55.053334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:55.053334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:55.053334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:55.053660 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:55.064104 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:55.065769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:55.074932 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1409/2825] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 10:53:55.527674 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:53:55.530025 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:55.607630 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764132b-2114-44e6-95ba-f14036cb0fa1 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:53:55.612740 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b059cb84-04ae-4eb7-86df-c536af33140f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:53:55.613430 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059cb84-04ae-4eb7-86df-c536af33140f service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:53:55.613715 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059cb84-04ae-4eb7-86df-c536af33140f service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:53:55.613986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b059cb84-04ae-4eb7-86df-c536af33140f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:53:55.615744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-b059cb84-04ae-4eb7-86df-c536af33140f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:53:55.629300 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059cb84-04ae-4eb7-86df-c536af33140f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:53:55.631022 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b059cb84-04ae-4eb7-86df-c536af33140f service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:53:55.649306 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1417/2826] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:53:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:54:05.070519 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=64 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:05.637645 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=67 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:07.114096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:54:07.129138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347aad7f-0828-46cf-924d-50620ab454b8 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:54:07.134650 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:54:07.135325 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:54:07.135609 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:54:07.135895 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:54:07.137662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:54:07.150199 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:54:07.151731 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:54:07.162647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1410/2827] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:54:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:54:17.157927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:54:58.754590 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ba100-04ea-407a-8c53-dca71d7b2a38 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:54:58.755141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ba100-04ea-407a-8c53-dca71d7b2a38 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:54:58.755141 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ba100-04ea-407a-8c53-dca71d7b2a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:54:58.772913 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-379ba100-04ea-407a-8c53-dca71d7b2a38 None None] MFA Rules not processed for user `ea49ebe4f80d48399086cacfe7b4bfdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:54:58.774066 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-379ba100-04ea-407a-8c53-dca71d7b2a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:54:58.781371 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1418/2828] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:54:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:55:08.767355 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=3 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:55:13.321976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:55:13.334347 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d640511-4bb7-46d8-ac1b-9e5ae79c2f6c service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:55:13.339941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8de75a1df6784111aabec60da40ab756', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:55:13.342673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:55:13.342673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:55:13.342673 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:55:13.343180 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:55:13.356989 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:55:13.358737 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:55:13.369542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1411/2829] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:55:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:55:23.364938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=6 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:56:04.531916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:56:04.532478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:56:04.532478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:56:04.550461 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] MFA Rules not processed for user `ea49ebe4f80d48399086cacfe7b4bfdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:56:04.551701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:56:04.572690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1419/2830] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:56:14.566855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=9 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:57:05.948094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:57:05.969101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df92cd-c395-4d4e-a4d6-981455c2188f service nova] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:57:05.972794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81733b0f-7968-4d55-bbac-910bd5a4cfef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:57:05.973499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81733b0f-7968-4d55-bbac-910bd5a4cfef service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:57:05.973785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81733b0f-7968-4d55-bbac-910bd5a4cfef service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:57:05.974063 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81733b0f-7968-4d55-bbac-910bd5a4cfef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:57:05.975827 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-81733b0f-7968-4d55-bbac-910bd5a4cfef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:57:05.996175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81733b0f-7968-4d55-bbac-910bd5a4cfef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:57:05.997639 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81733b0f-7968-4d55-bbac-910bd5a4cfef service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:57:06.006307 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1412/2831] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:57:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:57:06.985794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:57:06.985794 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:57:07.018642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc062a4-d21f-479c-9911-875bd63955e5 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:57:07.024052 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:57:07.024506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:57:07.024792 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:57:07.025072 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:57:07.026925 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:57:07.040491 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:57:07.042160 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:57:07.052888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1420/2832] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:57:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:57:14.615861 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5363602-0c57-4a28-ae18-4759d4856913 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:57:14.616374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5363602-0c57-4a28-ae18-4759d4856913 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:57:14.616488 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5363602-0c57-4a28-ae18-4759d4856913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:57:14.634653 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-d5363602-0c57-4a28-ae18-4759d4856913 None None] MFA Rules not processed for user `ea49ebe4f80d48399086cacfe7b4bfdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:57:14.635887 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-d5363602-0c57-4a28-ae18-4759d4856913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:57:14.643897 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1413/2833] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:57:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:57:17.049010 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=15 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:57:24.629299 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=18 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:28.271214 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:28.283600 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bc8ad-a5a6-435c-bec4-061107323064 service placement] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:28.287978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:28.288849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:28.289129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:28.289435 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] PATH_INFO: `/v3/users/ea49ebe4f80d48399086cacfe7b4bfdb` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:28.290642 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea49ebe4f80d48399086cacfe7b4bfdb)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:28.292571 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:28.459567 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea49ebe4f80d48399086cacfe7b4bfdb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:28.466432 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea49ebe4f80d48399086cacfe7b4bfdb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:28.467092 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea49ebe4f80d48399086cacfe7b4bfdb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:28.478921 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea49ebe4f80d48399086cacfe7b4bfdb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:28.542731 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-a7370c69-fae3-445e-8970-4ba4c6374dd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea49ebe4f80d48399086cacfe7b4bfdb'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:28.549471 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1421/2834] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:58:28 2022] DELETE /identity/v3/users/ea49ebe4f80d48399086cacfe7b4bfdb => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:58:28.778630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5363602-0c57-4a28-ae18-4759d4856913 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:28.795406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5363602-0c57-4a28-ae18-4759d4856913 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:28.846848 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:28.847713 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:28.848013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:28.848289 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] PATH_INFO: `/v3/projects/c94763dacd084cd69ee0729ee21d28a2` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:28.849499 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c94763dacd084cd69ee0729ee21d28a2)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:28.851396 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:28.956763 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c94763dacd084cd69ee0729ee21d28a2', 'request_id': 'req-19bf4471-a093-4512-b979-a6c44a43a9f1'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:29.012080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c94763dacd084cd69ee0729ee21d28a2', 'request_id': 'req-19bf4471-a093-4512-b979-a6c44a43a9f1'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:29.018785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c94763dacd084cd69ee0729ee21d28a2', 'request_id': 'req-19bf4471-a093-4512-b979-a6c44a43a9f1'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:29.030251 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] The token cache is being invalidate because project c94763dacd084cd69ee0729ee21d28a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:58:29.030559 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:29.031207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-19bf4471-a093-4512-b979-a6c44a43a9f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:58:29.032738 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1414/2835] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:58:28 2022] DELETE /identity/v3/projects/c94763dacd084cd69ee0729ee21d28a2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:58:29.037818 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d673c9-26ce-42c4-9dda-1394cd924151 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.044784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d673c9-26ce-42c4-9dda-1394cd924151 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.044784 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d673c9-26ce-42c4-9dda-1394cd924151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.056320 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-66d673c9-26ce-42c4-9dda-1394cd924151 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:58:29.057478 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-66d673c9-26ce-42c4-9dda-1394cd924151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:58:29.116067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1422/2836] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:58:29.123709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aad35-9dbb-4761-b48f-87ecaae121ea None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.123997 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aad35-9dbb-4761-b48f-87ecaae121ea None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.124820 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805aad35-9dbb-4761-b48f-87ecaae121ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.144345 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-805aad35-9dbb-4761-b48f-87ecaae121ea None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:58:29.145689 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-805aad35-9dbb-4761-b48f-87ecaae121ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:58:29.153847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1415/2837] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:58:29.158192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d673c9-26ce-42c4-9dda-1394cd924151 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.168228 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d673c9-26ce-42c4-9dda-1394cd924151 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.173504 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.174166 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.174468 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.174732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.175469 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:58:29.175780 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.177417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:29.189620 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1423/2838] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:58:29.194048 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805aad35-9dbb-4761-b48f-87ecaae121ea None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.203314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805aad35-9dbb-4761-b48f-87ecaae121ea None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.208173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.208818 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.209100 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.209414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.210101 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.212194 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:29.305221 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1416/2839] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:58:29 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:58:29.309330 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.320210 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27121efb-6935-4aaf-a2c6-f00382ff3c93 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.376589 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.377409 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.377695 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.377973 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.378658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.380610 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:29.387132 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:29.446885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1424/2840] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:58:29 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:58:29.450969 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.461785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5e06df-0c0a-40e0-9230-7ef3c2cf8b0a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.466561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.470391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.470391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.470391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.470391 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.470960 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:29.480927 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1417/2841] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:58:29.485750 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.496242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7998b-f941-400f-b5ad-0a167a6ce779 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.500975 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606dff9-e7f2-4cff-bd93-4a8ddf83c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.501665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606dff9-e7f2-4cff-bd93-4a8ddf83c72b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.501946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606dff9-e7f2-4cff-bd93-4a8ddf83c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.502221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606dff9-e7f2-4cff-bd93-4a8ddf83c72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.502969 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606dff9-e7f2-4cff-bd93-4a8ddf83c72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.504552 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606dff9-e7f2-4cff-bd93-4a8ddf83c72b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:29.514294 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1425/2842] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:58:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:58:29.518592 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.528541 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ffd72a-ec72-4351-8e95-dba905418bb8 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.533928 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.534813 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.535094 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.535374 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] PATH_INFO: `/v3/projects/7019195404aa4be4b7f8c6cd14dd6554/users/b5553745163f42da8bf3e84490da0e30/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.550698 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7019195404aa4be4b7f8c6cd14dd6554, user_id=b5553745163f42da8bf3e84490da0e30, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.553363 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:29.635197 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1418/2843] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:58:29 2022] PUT /identity/v3/projects/7019195404aa4be4b7f8c6cd14dd6554/users/b5553745163f42da8bf3e84490da0e30/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:58:29.645963 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820596c1-123c-4150-aa36-7614759fc7ad None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.646251 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820596c1-123c-4150-aa36-7614759fc7ad None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.646529 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820596c1-123c-4150-aa36-7614759fc7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.673387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-820596c1-123c-4150-aa36-7614759fc7ad None None] MFA Rules not processed for user `b5553745163f42da8bf3e84490da0e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:58:29.674575 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-820596c1-123c-4150-aa36-7614759fc7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:58:29.738231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1426/2844] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:58:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:58:29.742993 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.772580 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:29.804207 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128a28-1c0f-43eb-8d37-418d85dcd4d1 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.808774 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.811003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.811003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.811003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.811003 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.812734 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:29.888383 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1419/2845] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:58:29 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:58:29.892968 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820596c1-123c-4150-aa36-7614759fc7ad None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:29.903924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820596c1-123c-4150-aa36-7614759fc7ad None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:29.960516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:29.961188 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:29.961505 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:29.961788 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:29.962486 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:29.964401 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:30.041063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1427/2846] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:58:29 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:58:30.045039 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:30.055587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d316d-441f-42d3-b70a-f35f17da88f5 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:30.060154 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:30.060810 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:30.061096 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:30.061420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:30.062339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:30.063941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:30.075443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1420/2847] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:58:30.083718 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:30.094147 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4828d17-85f9-4eb3-abe3-69d5a90c98a5 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:30.098690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:30.099563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:30.099844 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:30.100115 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] PATH_INFO: `/v3/projects/0bc1fd4e933f42e9858687f5b849f9c6/users/d69a39a82ae940abab52c9c7328685e8/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:30.114929 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc1fd4e933f42e9858687f5b849f9c6, user_id=d69a39a82ae940abab52c9c7328685e8, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:30.117525 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:30.190016 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1428/2848] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:58:30 2022] PUT /identity/v3/projects/0bc1fd4e933f42e9858687f5b849f9c6/users/d69a39a82ae940abab52c9c7328685e8/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 10:58:30.194097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:30.204677 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e187719-e367-4dd8-a5b7-b2ff2a522f95 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:30.259057 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf2f0e-c740-4f90-8c5a-2353f4749aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:30.259715 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2f0e-c740-4f90-8c5a-2353f4749aba admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:30.260000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2f0e-c740-4f90-8c5a-2353f4749aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:30.260276 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf2f0e-c740-4f90-8c5a-2353f4749aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:30.260992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf2f0e-c740-4f90-8c5a-2353f4749aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:30.262630 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf2f0e-c740-4f90-8c5a-2353f4749aba admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:30.272658 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1421/2849] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:58:30.276828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:30.287203 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5542e1f-8370-454f-b418-839b154cddb0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:30.291717 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:30.292585 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:30.292889 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:30.293161 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] PATH_INFO: `/v3/projects/0bc1fd4e933f42e9858687f5b849f9c6/users/d69a39a82ae940abab52c9c7328685e8/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:30.295119 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc1fd4e933f42e9858687f5b849f9c6, user_id=d69a39a82ae940abab52c9c7328685e8, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:30.297842 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:30.307799 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1429/2850] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:58:30 2022] PUT /identity/v3/projects/0bc1fd4e933f42e9858687f5b849f9c6/users/d69a39a82ae940abab52c9c7328685e8/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:58:30.318373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271d1e4-642d-4776-9f96-527ec501b880 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:30.318666 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271d1e4-642d-4776-9f96-527ec501b880 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:30.318946 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271d1e4-642d-4776-9f96-527ec501b880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:30.345348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-5271d1e4-642d-4776-9f96-527ec501b880 None None] MFA Rules not processed for user `d69a39a82ae940abab52c9c7328685e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:58:30.346569 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-5271d1e4-642d-4776-9f96-527ec501b880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:58:30.413710 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1422/2851] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:58:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:58:30.424843 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:30.426533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:58:30.448712 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:30.485595 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43deab9b-0254-4217-b97e-b831d5711d73 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:30.489242 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:30.489947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:30.490229 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:30.490506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:30.500426 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:30.501901 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:30.511670 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1430/2852] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:58:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:58:30.694885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271d1e4-642d-4776-9f96-527ec501b880 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:58:30.704912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271d1e4-642d-4776-9f96-527ec501b880 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:58:30.708668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43a9f02-9c37-4873-92d2-83bad17e076e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:58:30.709334 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a9f02-9c37-4873-92d2-83bad17e076e service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:58:30.709615 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a9f02-9c37-4873-92d2-83bad17e076e service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:58:30.709890 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43a9f02-9c37-4873-92d2-83bad17e076e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:58:30.728443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:30.766177 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a9f02-9c37-4873-92d2-83bad17e076e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:58:30.767951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43a9f02-9c37-4873-92d2-83bad17e076e service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:58:30.777449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1423/2853] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:58:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:58:40.595514 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=17 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:58:40.823528 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=18 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:18.428825 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:18.489153 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46201a8b-d5ca-4194-a42c-939a668f4cbf service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:18.493647 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:18.494513 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:18.494789 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:18.495057 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] PATH_INFO: `/v3/users/b5553745163f42da8bf3e84490da0e30` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:18.496866 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5553745163f42da8bf3e84490da0e30)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:18.498802 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:18.680915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5553745163f42da8bf3e84490da0e30'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:18.687983 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5553745163f42da8bf3e84490da0e30'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:18.688758 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5553745163f42da8bf3e84490da0e30'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:18.700658 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5553745163f42da8bf3e84490da0e30'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:18.788704 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5553745163f42da8bf3e84490da0e30'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:18.795593 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1431/2854] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:59:18 2022] DELETE /identity/v3/users/b5553745163f42da8bf3e84490da0e30 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:18.800185 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:18.800656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:18.800765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:18.828972 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:18.830219 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:18.889256 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1424/2855] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:18.899077 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:18.900454 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:18.959248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67f630-16b8-4b15-80fc-baf3f672744b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:18.962971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:18.963622 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:18.963909 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:18.964187 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:18.974288 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:18.978826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:18.985723 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1432/2856] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:59:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:59:19.350357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:19.351800 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:19.364132 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc2940-5bb8-4718-bc86-73f92896a82e None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:19.368728 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:19.369628 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:19.369908 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:19.370187 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] PATH_INFO: `/v3/projects/7019195404aa4be4b7f8c6cd14dd6554` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:19.371372 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7019195404aa4be4b7f8c6cd14dd6554)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:19.373847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:19.464137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7019195404aa4be4b7f8c6cd14dd6554', 'request_id': 'req-286e7dec-437d-49bd-abe0-682448ecf889'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.514682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7019195404aa4be4b7f8c6cd14dd6554', 'request_id': 'req-286e7dec-437d-49bd-abe0-682448ecf889'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.520461 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7019195404aa4be4b7f8c6cd14dd6554', 'request_id': 'req-286e7dec-437d-49bd-abe0-682448ecf889'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.532158 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] The token cache is being invalidate because project 7019195404aa4be4b7f8c6cd14dd6554 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:59:19.532588 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.533165 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.534851 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1425/2857] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:59:19 2022] DELETE /identity/v3/projects/7019195404aa4be4b7f8c6cd14dd6554 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:19.538757 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:19.549120 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351c82e-ab07-4cc7-a3ed-26f68fb7b1a2 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:19.566456 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:19.604271 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:19.605150 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:19.605625 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:19.607003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] PATH_INFO: `/v3/users/d69a39a82ae940abab52c9c7328685e8` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:19.608226 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d69a39a82ae940abab52c9c7328685e8)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:19.610172 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:19.801819 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd69a39a82ae940abab52c9c7328685e8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.808562 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd69a39a82ae940abab52c9c7328685e8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.809404 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd69a39a82ae940abab52c9c7328685e8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.821351 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd69a39a82ae940abab52c9c7328685e8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.895915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-66097860-396f-4dba-9032-7736fbe62903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd69a39a82ae940abab52c9c7328685e8'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:19.902372 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1433/2858] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:59:19 2022] DELETE /identity/v3/users/d69a39a82ae940abab52c9c7328685e8 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:20.319865 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:20.323228 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:20.336115 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e7dec-437d-49bd-abe0-682448ecf889 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:20.392215 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:20.393062 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:20.393371 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:20.393649 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] PATH_INFO: `/v3/projects/0bc1fd4e933f42e9858687f5b849f9c6` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:20.394794 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc1fd4e933f42e9858687f5b849f9c6)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:20.396684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:20.412679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:20.520554 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc1fd4e933f42e9858687f5b849f9c6', 'request_id': 'req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:20.593077 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc1fd4e933f42e9858687f5b849f9c6', 'request_id': 'req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:20.598881 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc1fd4e933f42e9858687f5b849f9c6', 'request_id': 'req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:20.610544 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] The token cache is being invalidate because project 0bc1fd4e933f42e9858687f5b849f9c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:59:20.610831 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:20.611573 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-590903f0-ffb1-4061-b7ff-d5fb5eae54bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:20.613190 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1426/2859] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:59:20 2022] DELETE /identity/v3/projects/0bc1fd4e933f42e9858687f5b849f9c6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:20.618018 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188b34e-bb64-43f2-9c3f-b3c16ad65080 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:20.618506 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188b34e-bb64-43f2-9c3f-b3c16ad65080 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:20.618624 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188b34e-bb64-43f2-9c3f-b3c16ad65080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:20.637691 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-3188b34e-bb64-43f2-9c3f-b3c16ad65080 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:20.638906 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-3188b34e-bb64-43f2-9c3f-b3c16ad65080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:20.699549 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1434/2860] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:20.707348 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b121e-b00d-4432-bafc-ce85360b8697 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:20.707637 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b121e-b00d-4432-bafc-ce85360b8697 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:20.708173 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b121e-b00d-4432-bafc-ce85360b8697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:20.727481 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-9b1b121e-b00d-4432-bafc-ce85360b8697 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:20.728695 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9b1b121e-b00d-4432-bafc-ce85360b8697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:20.737542 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1427/2861] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:20.741768 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188b34e-bb64-43f2-9c3f-b3c16ad65080 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:20.752123 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188b34e-bb64-43f2-9c3f-b3c16ad65080 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:20.756665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:20.757395 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:20.757662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:20.757935 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:20.758657 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:59:20.758959 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:20.760557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:20.772244 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1435/2862] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:59:20.776431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b121e-b00d-4432-bafc-ce85360b8697 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:20.786392 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1b121e-b00d-4432-bafc-ce85360b8697 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:20.791036 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:20.791685 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:20.791968 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:20.792245 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:20.792941 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:20.795664 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:20.896952 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1428/2863] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:59:20 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:59:20.900986 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:20.911980 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab1d91-8ea2-4fb3-afd6-0491e05ac14b admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:20.942664 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:20.972652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:20.973440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:20.973724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:20.974004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:20.974701 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:20.976796 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:21.109828 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1436/2864] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:59:20 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:59:21.114028 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:21.124216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3ab3-66d7-43d5-b488-4ad4370f4b51 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:21.128846 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:21.129529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.129811 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.130087 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.130804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:21.132384 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:21.143102 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1429/2865] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:59:21.147189 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:21.157687 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e334111-973b-4751-8ab1-55158a3cb8ed admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:21.162289 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8927f2c9-5ed6-41d9-9c25-da73eaf77afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:21.162946 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f2c9-5ed6-41d9-9c25-da73eaf77afd admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.163224 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f2c9-5ed6-41d9-9c25-da73eaf77afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.163498 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927f2c9-5ed6-41d9-9c25-da73eaf77afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.164222 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927f2c9-5ed6-41d9-9c25-da73eaf77afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:21.165851 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8927f2c9-5ed6-41d9-9c25-da73eaf77afd admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:21.176457 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1437/2866] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 10:59:21.180612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:21.190892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80df475-a452-4f62-b8eb-5a58f809c7aa admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:21.195326 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:21.196193 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.196474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.196750 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] PATH_INFO: `/v3/projects/3f1ae5d6b96e4b3c977ee6ecccdc36a4/users/926263c083964cd3a647a7d459901084/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.212566 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f1ae5d6b96e4b3c977ee6ecccdc36a4, user_id=926263c083964cd3a647a7d459901084, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:21.215990 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:21.320898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1430/2867] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:59:21 2022] PUT /identity/v3/projects/3f1ae5d6b96e4b3c977ee6ecccdc36a4/users/926263c083964cd3a647a7d459901084/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:21.333681 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda7f8d-ca97-42ad-bb3e-03a5381edf14 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.333971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda7f8d-ca97-42ad-bb3e-03a5381edf14 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.334250 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda7f8d-ca97-42ad-bb3e-03a5381edf14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.360473 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-4bda7f8d-ca97-42ad-bb3e-03a5381edf14 None None] MFA Rules not processed for user `926263c083964cd3a647a7d459901084`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:21.361711 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-4bda7f8d-ca97-42ad-bb3e-03a5381edf14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:21.423887 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1438/2868] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:21 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:21.428386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:21.489373 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb80d39a-ae76-42df-a573-606fbb8b0264 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:21.493935 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:21.494591 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.494873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.495155 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.495832 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:21.497854 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:21.622138 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1431/2869] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:59:21 2022] POST /identity/v3/projects => generated 363 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:59:21.626422 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda7f8d-ca97-42ad-bb3e-03a5381edf14 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:21.637158 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda7f8d-ca97-42ad-bb3e-03a5381edf14 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:21.694174 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:21.694855 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.695138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.695415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.696104 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:21.698048 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:21.829772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1439/2870] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:59:21 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:59:21.834010 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:21.844386 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd4100-a6e7-4f5d-932f-34c021a46e58 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:21.848859 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:21.849547 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.849833 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.850108 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.850837 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:21.852427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:21.862912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1432/2871] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:59:21.866916 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:21.877665 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8b467-2304-4d11-bdea-7a4ac11fbb65 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:21.882403 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:21.883315 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:21.883597 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:21.883877 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] PATH_INFO: `/v3/projects/f2f26c6f5290498a857eb6fb4055f38f/users/83c3281409fc48969105ef41931ef9bd/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:21.899124 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f26c6f5290498a857eb6fb4055f38f, user_id=83c3281409fc48969105ef41931ef9bd, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:21.902391 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:22.004678 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1440/2872] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:59:21 2022] PUT /identity/v3/projects/f2f26c6f5290498a857eb6fb4055f38f/users/83c3281409fc48969105ef41931ef9bd/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:22.008513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:22.018799 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43c812-1955-4d86-a368-b923b85fd7f2 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:22.030651 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:22.074829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c9661-c1f6-4b44-9950-0185c7a2ac8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:22.075662 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c9661-c1f6-4b44-9950-0185c7a2ac8b admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:22.075942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c9661-c1f6-4b44-9950-0185c7a2ac8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:22.076216 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c9661-c1f6-4b44-9950-0185c7a2ac8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:22.076938 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c9661-c1f6-4b44-9950-0185c7a2ac8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:22.078612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c9661-c1f6-4b44-9950-0185c7a2ac8b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:22.088418 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1433/2873] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:59:22.092767 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:22.098308 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:22.104542 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75232b8c-3b97-4b44-a8c2-e3b808453f1c admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:22.110177 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:22.111054 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:22.111341 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:22.111619 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] PATH_INFO: `/v3/projects/f2f26c6f5290498a857eb6fb4055f38f/users/83c3281409fc48969105ef41931ef9bd/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:22.113608 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f26c6f5290498a857eb6fb4055f38f, user_id=83c3281409fc48969105ef41931ef9bd, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:22.116217 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:22.126996 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1441/2874] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:59:22 2022] PUT /identity/v3/projects/f2f26c6f5290498a857eb6fb4055f38f/users/83c3281409fc48969105ef41931ef9bd/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:22.138385 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0194b4c-9c9c-4f60-a6e7-7cc0386c0802 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:22.138704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0194b4c-9c9c-4f60-a6e7-7cc0386c0802 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:22.138984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0194b4c-9c9c-4f60-a6e7-7cc0386c0802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:22.165641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-a0194b4c-9c9c-4f60-a6e7-7cc0386c0802 None None] MFA Rules not processed for user `83c3281409fc48969105ef41931ef9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:22.166640 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-a0194b4c-9c9c-4f60-a6e7-7cc0386c0802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:22.235189 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1434/2875] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:22 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:22.246512 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:22.248759 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:22.313211 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07b408-afd8-4b33-8f05-0adf70d11438 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:22.316774 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:22.317448 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:22.317730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:22.318004 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:22.326915 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:22.328336 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:22.337834 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1442/2876] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:59:22 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:59:22.538466 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0194b4c-9c9c-4f60-a6e7-7cc0386c0802 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:22.549024 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0194b4c-9c9c-4f60-a6e7-7cc0386c0802 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:22.552585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae67f3e-4b22-48a7-b3fd-838610faf040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:22.553284 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae67f3e-4b22-48a7-b3fd-838610faf040 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:22.553596 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae67f3e-4b22-48a7-b3fd-838610faf040 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:22.553876 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae67f3e-4b22-48a7-b3fd-838610faf040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:22.611080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae67f3e-4b22-48a7-b3fd-838610faf040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:22.612570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae67f3e-4b22-48a7-b3fd-838610faf040 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:22.621317 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1435/2877] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:59:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:59:32.333675 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=34,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:32.804339 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=84 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:34.270408 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:34.331636 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af51e6-6ea9-4289-8ae5-a6aea7604e4b service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:34.336483 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:34.337382 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:34.337671 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:34.337972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] PATH_INFO: `/v3/users/926263c083964cd3a647a7d459901084` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:34.339193 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=926263c083964cd3a647a7d459901084)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:34.341113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:34.515822 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926263c083964cd3a647a7d459901084'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:34.522902 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926263c083964cd3a647a7d459901084'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:34.523573 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926263c083964cd3a647a7d459901084'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:34.535293 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926263c083964cd3a647a7d459901084'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:34.603941 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926263c083964cd3a647a7d459901084'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:34.610596 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1443/2878] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:59:34 2022] DELETE /identity/v3/users/926263c083964cd3a647a7d459901084 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:34.615050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:34.615529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:34.615610 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:34.632797 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:34.634027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:34.691550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1436/2879] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:34.701023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:34.702699 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:34.759561 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa7e2c-c600-4932-acc4-be4345691f92 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:34.763170 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:34.763807 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:34.764087 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:34.764356 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:34.773724 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:34.775131 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:34.784885 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1444/2880] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:59:35.219585 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:35.221018 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:35.233529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4770a7-72f0-4041-aa43-636a18c3b8c5 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:35.238431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:35.239297 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:35.239583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:35.239864 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] PATH_INFO: `/v3/projects/3f1ae5d6b96e4b3c977ee6ecccdc36a4` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:35.241218 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f1ae5d6b96e4b3c977ee6ecccdc36a4)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:35.243151 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:35.360873 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f1ae5d6b96e4b3c977ee6ecccdc36a4', 'request_id': 'req-1f2a2012-be72-4fe1-91f1-092a165815bd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.434234 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f1ae5d6b96e4b3c977ee6ecccdc36a4', 'request_id': 'req-1f2a2012-be72-4fe1-91f1-092a165815bd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.440676 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f1ae5d6b96e4b3c977ee6ecccdc36a4', 'request_id': 'req-1f2a2012-be72-4fe1-91f1-092a165815bd'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.451714 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] The token cache is being invalidate because project 3f1ae5d6b96e4b3c977ee6ecccdc36a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:59:35.452570 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.453205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.454779 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1437/2881] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:59:35 2022] DELETE /identity/v3/projects/3f1ae5d6b96e4b3c977ee6ecccdc36a4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:35.458939 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:35.468846 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6cae58-8300-4ace-a8d5-f0179cd23aa2 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:35.486458 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:35.524910 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:35.525803 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:35.526084 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:35.526360 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] PATH_INFO: `/v3/users/83c3281409fc48969105ef41931ef9bd` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:35.527563 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83c3281409fc48969105ef41931ef9bd)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:35.529467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:35.709011 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83c3281409fc48969105ef41931ef9bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.715696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83c3281409fc48969105ef41931ef9bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.716349 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83c3281409fc48969105ef41931ef9bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.728530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83c3281409fc48969105ef41931ef9bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.809735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-6b39cdf7-02a1-4a02-a22e-5466a1f3e505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83c3281409fc48969105ef41931ef9bd'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:35.816238 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1445/2882] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 10:59:35 2022] DELETE /identity/v3/users/83c3281409fc48969105ef41931ef9bd => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:36.257683 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:36.259106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:36.273259 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a2012-be72-4fe1-91f1-092a165815bd admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:36.331097 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:36.331955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:36.332237 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:36.332510 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] PATH_INFO: `/v3/projects/f2f26c6f5290498a857eb6fb4055f38f` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:36.334013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f26c6f5290498a857eb6fb4055f38f)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:36.335909 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:36.423212 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:36.439513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f26c6f5290498a857eb6fb4055f38f', 'request_id': 'req-5583d212-ca36-47d8-a671-97cf7ea637e6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:36.524222 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f26c6f5290498a857eb6fb4055f38f', 'request_id': 'req-5583d212-ca36-47d8-a671-97cf7ea637e6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:36.529912 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f26c6f5290498a857eb6fb4055f38f', 'request_id': 'req-5583d212-ca36-47d8-a671-97cf7ea637e6'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:36.541805 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] The token cache is being invalidate because project f2f26c6f5290498a857eb6fb4055f38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 10:59:36.542093 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:36.542724 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-5583d212-ca36-47d8-a671-97cf7ea637e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 10:59:36.546427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1438/2883] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 10:59:36 2022] DELETE /identity/v3/projects/f2f26c6f5290498a857eb6fb4055f38f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:36.549888 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ec20-5b62-499d-8386-7415633bab5e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:36.550324 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ec20-5b62-499d-8386-7415633bab5e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:36.550442 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f740ec20-5b62-499d-8386-7415633bab5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:36.568634 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f740ec20-5b62-499d-8386-7415633bab5e None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:36.569873 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f740ec20-5b62-499d-8386-7415633bab5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:36.627947 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1446/2884] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:36.635688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276ae5e-b060-4fa0-a7e9-ecf00c59fe29 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:36.636052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276ae5e-b060-4fa0-a7e9-ecf00c59fe29 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:36.636353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276ae5e-b060-4fa0-a7e9-ecf00c59fe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:36.654627 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-0276ae5e-b060-4fa0-a7e9-ecf00c59fe29 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:36.656380 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-0276ae5e-b060-4fa0-a7e9-ecf00c59fe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:36.664406 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1439/2885] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:36.668480 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ec20-5b62-499d-8386-7415633bab5e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:36.678357 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f740ec20-5b62-499d-8386-7415633bab5e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:36.683221 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:36.683878 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:36.684162 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:36.684440 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:36.685168 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 10:59:36.685516 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:36.687149 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:36.699554 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1447/2886] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 10:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 10:59:36.704766 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276ae5e-b060-4fa0-a7e9-ecf00c59fe29 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:36.714147 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276ae5e-b060-4fa0-a7e9-ecf00c59fe29 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:36.719474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:36.720117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:36.720398 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:36.720674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:36.721397 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:36.723301 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:36.816572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1440/2887] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:59:36 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:59:36.820702 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:36.831354 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c871b7-c18e-478f-9a63-e70c730b6a5f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:36.857533 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:36.887327 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:36.888005 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:36.888291 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:36.888568 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:36.889322 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:36.892157 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.047013 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1448/2888] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:59:36 2022] POST /identity/v3/users => generated 288 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:59:37.051602 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.062250 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a59154-42d9-4ae6-949b-e92df45e2268 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.066777 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.067370 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.067647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.067922 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.068669 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.070275 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.082052 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1441/2889] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:59:37.089588 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.096889 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c119376-4060-4fa3-8774-b14bf4dbbf09 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.101612 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715b09bf-dd3f-4cfb-bcf2-d9353b80c76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.102266 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b09bf-dd3f-4cfb-bcf2-d9353b80c76c admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.102550 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b09bf-dd3f-4cfb-bcf2-d9353b80c76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.102823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715b09bf-dd3f-4cfb-bcf2-d9353b80c76c admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.103547 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b09bf-dd3f-4cfb-bcf2-d9353b80c76c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.105133 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715b09bf-dd3f-4cfb-bcf2-d9353b80c76c admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.116362 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1449/2890] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:59:37.120622 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.130856 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d166c-517c-444a-b59a-b3cee024c038 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.136945 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.137885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.138166 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.138439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] PATH_INFO: `/v3/projects/1b60bfdf461b4fe0ae1478b211de5c13/users/12baa2642d244142842fcd4870663e8a/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.153341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b60bfdf461b4fe0ae1478b211de5c13, user_id=12baa2642d244142842fcd4870663e8a, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.155974 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.223668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1442/2891] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:59:37 2022] PUT /identity/v3/projects/1b60bfdf461b4fe0ae1478b211de5c13/users/12baa2642d244142842fcd4870663e8a/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:37.235736 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd776-f9e5-4579-9032-7ffdd07ceb2e None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.236026 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd776-f9e5-4579-9032-7ffdd07ceb2e None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.236304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226cd776-f9e5-4579-9032-7ffdd07ceb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.262459 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-226cd776-f9e5-4579-9032-7ffdd07ceb2e None None] MFA Rules not processed for user `12baa2642d244142842fcd4870663e8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:37.263662 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-226cd776-f9e5-4579-9032-7ffdd07ceb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:37.329425 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1450/2892] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:37.334044 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.396220 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36573896-9695-443e-9bdb-e078af663f83 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.400878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.401563 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.401853 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.402137 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.402838 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.404740 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.509796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1443/2893] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 10:59:37 2022] POST /identity/v3/projects => generated 377 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 10:59:37.513872 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd776-f9e5-4579-9032-7ffdd07ceb2e None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.525215 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226cd776-f9e5-4579-9032-7ffdd07ceb2e None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.580652 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.581307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.581548 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.581824 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.582683 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.584850 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.657521 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1451/2894] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 10:59:37 2022] POST /identity/v3/users => generated 295 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 10:59:37.661389 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.671607 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686e1f-61be-4344-aca6-868a791a2730 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.676930 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.677620 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.677901 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.678175 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.678892 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.680474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.690991 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1444/2895] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:59:37.695143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.705421 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae63aab-3fdc-4b48-bf72-30d047f6c68f admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.710180 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.711059 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.711340 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.711614 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] PATH_INFO: `/v3/projects/9ad4b0490f9c4c9a9dcbf1110bdf9d64/users/9aa0c41cbcba479c80ed0278ad02b480/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.726786 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad4b0490f9c4c9a9dcbf1110bdf9d64, user_id=9aa0c41cbcba479c80ed0278ad02b480, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.729462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.822919 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1452/2896] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:59:37 2022] PUT /identity/v3/projects/9ad4b0490f9c4c9a9dcbf1110bdf9d64/users/9aa0c41cbcba479c80ed0278ad02b480/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:37.826769 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.845027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a1bffb-cb63-4e24-afb2-9739e6ce5c96 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.856246 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:37.891463 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd9858-122f-46cf-a36c-9fadc203bc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.892170 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd9858-122f-46cf-a36c-9fadc203bc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.892414 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd9858-122f-46cf-a36c-9fadc203bc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.892684 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd9858-122f-46cf-a36c-9fadc203bc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.893443 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd9858-122f-46cf-a36c-9fadc203bc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.895020 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd9858-122f-46cf-a36c-9fadc203bc46 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.904688 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1445/2897] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 10:59:37 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 10:59:37.909490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:37.914955 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:37.920280 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918422d8-8e0c-4815-9d8f-122867527954 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:37.925924 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:37.926826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.927113 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.927392 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] PATH_INFO: `/v3/projects/9ad4b0490f9c4c9a9dcbf1110bdf9d64/users/9aa0c41cbcba479c80ed0278ad02b480/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.929398 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ad4b0490f9c4c9a9dcbf1110bdf9d64, user_id=9aa0c41cbcba479c80ed0278ad02b480, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:37.932071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:37.942417 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1453/2898] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 10:59:37 2022] PUT /identity/v3/projects/9ad4b0490f9c4c9a9dcbf1110bdf9d64/users/9aa0c41cbcba479c80ed0278ad02b480/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 10:59:37.952474 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:37.952787 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:37.953068 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:37.980140 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] MFA Rules not processed for user `9aa0c41cbcba479c80ed0278ad02b480`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 10:59:37.981357 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:38.049704 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1446/2899] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 10:59:37 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 10:59:38.060912 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:38.063273 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:38.120202 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec42b54-75a0-44c4-9bbe-550792d30b51 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:38.123714 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:38.124348 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:38.124626 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:38.124900 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:38.179728 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:38.181295 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:38.190747 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1454/2900] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:59:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:59:44.380652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 10:59:44.381139 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 10:59:44.393925 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a1138-4be1-452c-bf72-d951af0e624f None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 10:59:44.397593 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff1e279-9798-4e97-881b-849d7364e066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 10:59:44.398263 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1e279-9798-4e97-881b-849d7364e066 service cinder] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 10:59:44.398545 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1e279-9798-4e97-881b-849d7364e066 service cinder] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 10:59:44.398880 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff1e279-9798-4e97-881b-849d7364e066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 10:59:44.414546 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1e279-9798-4e97-881b-849d7364e066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 10:59:44.418726 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff1e279-9798-4e97-881b-849d7364e066 service cinder] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 10:59:44.429316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1447/2901] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 10:59:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 10:59:48.185982 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 10:59:54.424082 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=58 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:11.558894 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:11.621164 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b52885-5231-46f0-817b-baf5466c09b2 service cinder] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:11.625690 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:11.626566 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:11.626849 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:11.627125 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] PATH_INFO: `/v3/users/12baa2642d244142842fcd4870663e8a` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:11.629095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12baa2642d244142842fcd4870663e8a)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:11.631056 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:11.818604 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12baa2642d244142842fcd4870663e8a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:11.825305 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12baa2642d244142842fcd4870663e8a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:11.825981 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12baa2642d244142842fcd4870663e8a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:11.838281 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12baa2642d244142842fcd4870663e8a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:11.889186 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12baa2642d244142842fcd4870663e8a'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:11.895898 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1455/2902] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 11:00:11 2022] DELETE /identity/v3/users/12baa2642d244142842fcd4870663e8a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:11.900534 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:11.901019 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:11.901117 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:11.919205 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 11:00:11.920404 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:11.980088 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1448/2903] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 11:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 11:00:11.990543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:11.991978 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:12.047445 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b115af3-9a14-415f-b79d-0bfe730102c7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:12.051800 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:12.052447 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:12.052735 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:12.053009 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:12.061991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:12.063427 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:12.073629 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1456/2904] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 11:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 11:00:12.452412 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:12.453866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:12.466951 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9028441-b550-4dcb-a430-60cc165eae92 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:12.471906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:12.472888 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:12.473178 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:12.473487 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] PATH_INFO: `/v3/projects/1b60bfdf461b4fe0ae1478b211de5c13` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:12.475430 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b60bfdf461b4fe0ae1478b211de5c13)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:12.477434 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:12.603785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b60bfdf461b4fe0ae1478b211de5c13', 'request_id': 'req-9fd3b4e2-acfa-4328-94e4-218fb97d083b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:12.667992 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b60bfdf461b4fe0ae1478b211de5c13', 'request_id': 'req-9fd3b4e2-acfa-4328-94e4-218fb97d083b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:12.673749 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b60bfdf461b4fe0ae1478b211de5c13', 'request_id': 'req-9fd3b4e2-acfa-4328-94e4-218fb97d083b'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:12.685572 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] The token cache is being invalidate because project 1b60bfdf461b4fe0ae1478b211de5c13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 11:00:12.685866 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:12.686558 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:12.688123 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1449/2905] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 11:00:12 2022] DELETE /identity/v3/projects/1b60bfdf461b4fe0ae1478b211de5c13 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:12.692540 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:12.702107 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3184b26-2bff-4794-8322-a492bd4b1c94 service neutron] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:12.720346 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:12.758763 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:12.759639 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:12.759923 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:12.760200 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] PATH_INFO: `/v3/users/9aa0c41cbcba479c80ed0278ad02b480` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:12.761415 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa0c41cbcba479c80ed0278ad02b480)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:12.763387 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:13.006001 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa0c41cbcba479c80ed0278ad02b480'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.012826 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa0c41cbcba479c80ed0278ad02b480'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.014127 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa0c41cbcba479c80ed0278ad02b480'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.026990 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa0c41cbcba479c80ed0278ad02b480'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.085543 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-194db022-68f4-468b-bce3-1ee284182649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa0c41cbcba479c80ed0278ad02b480'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.092971 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1457/2906] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 11:00:12 2022] DELETE /identity/v3/users/9aa0c41cbcba479c80ed0278ad02b480 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:13.515080 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:13.516656 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:13.530638 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3b4e2-acfa-4328-94e4-218fb97d083b admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:13.592679 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:13.593587 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:13.593872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:13.594145 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] PATH_INFO: `/v3/projects/9ad4b0490f9c4c9a9dcbf1110bdf9d64` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:13.595452 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ad4b0490f9c4c9a9dcbf1110bdf9d64)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:13.597979 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:13.679324 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:13.695464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ad4b0490f9c4c9a9dcbf1110bdf9d64', 'request_id': 'req-4e865364-ac01-4a1c-9bcd-999088b25047'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.782746 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ad4b0490f9c4c9a9dcbf1110bdf9d64', 'request_id': 'req-4e865364-ac01-4a1c-9bcd-999088b25047'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.789796 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ad4b0490f9c4c9a9dcbf1110bdf9d64', 'request_id': 'req-4e865364-ac01-4a1c-9bcd-999088b25047'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.803135 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] The token cache is being invalidate because project 9ad4b0490f9c4c9a9dcbf1110bdf9d64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80404) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 11:00:13.803431 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.806994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-4e865364-ac01-4a1c-9bcd-999088b25047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:13.806994 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1450/2907] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 11:00:13 2022] DELETE /identity/v3/projects/9ad4b0490f9c4c9a9dcbf1110bdf9d64 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:13.811696 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79181e-0079-4d27-bd64-4f5e27da14f9 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:13.812165 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79181e-0079-4d27-bd64-4f5e27da14f9 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:13.812256 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79181e-0079-4d27-bd64-4f5e27da14f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:13.830737 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-fb79181e-0079-4d27-bd64-4f5e27da14f9 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 11:00:13.835743 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-fb79181e-0079-4d27-bd64-4f5e27da14f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:13.893708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1458/2908] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 11:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 11:00:13.901522 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d3387-1da7-42f2-9af6-35b78396c926 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:13.901815 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d3387-1da7-42f2-9af6-35b78396c926 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:13.902095 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759d3387-1da7-42f2-9af6-35b78396c926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:13.922344 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-759d3387-1da7-42f2-9af6-35b78396c926 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 11:00:13.923511 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-759d3387-1da7-42f2-9af6-35b78396c926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:13.932021 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1451/2909] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 11:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 11:00:13.936091 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79181e-0079-4d27-bd64-4f5e27da14f9 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:13.945827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb79181e-0079-4d27-bd64-4f5e27da14f9 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:13.950378 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:13.951027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:13.951307 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:13.951578 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:13.952302 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80405) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 24 11:00:13.952605 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:13.954231 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:13.966640 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1459/2910] 10.222.0.35 () {58 vars in 1152 bytes} [Tue May 24 11:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 24 11:00:13.970898 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d3387-1da7-42f2-9af6-35b78396c926 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:13.981410 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759d3387-1da7-42f2-9af6-35b78396c926 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:13.986255 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:13.986906 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:13.987300 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:13.987636 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:13.988550 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:13.990470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.067976 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1452/2911] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 11:00:13 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 11:00:14.071893 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:14.082651 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d61b5a-4b35-4b40-9561-8b608b822cc3 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:14.108944 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:14.140053 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:14.140734 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.141015 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.141328 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.142027 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:14.144733 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.272823 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1460/2912] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 11:00:14 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 11:00:14.277000 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:14.287765 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0f3ae-4feb-4ca3-bfd2-ad8099a52a22 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:14.292346 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:14.293013 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.293316 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.293594 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.294314 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:14.295902 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.306917 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1453/2913] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 11:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 11:00:14.310781 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:14.321299 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba74d7e-3790-48f2-9732-6b2aa37570cf admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:14.325827 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c865f25e-fdfe-4f52-bb5c-1ae70526f13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:14.326481 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865f25e-fdfe-4f52-bb5c-1ae70526f13e admin admin] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.326762 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865f25e-fdfe-4f52-bb5c-1ae70526f13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.327036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c865f25e-fdfe-4f52-bb5c-1ae70526f13e admin admin] PATH_INFO: `/v3/roles` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.327765 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865f25e-fdfe-4f52-bb5c-1ae70526f13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:14.329393 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c865f25e-fdfe-4f52-bb5c-1ae70526f13e admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.340329 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1461/2914] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 11:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 11:00:14.344427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:14.354860 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd07ca-901d-4895-aaeb-3de0c282985f admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:14.359470 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:14.360353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] REQUEST_METHOD: `PUT` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.360647 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.360915 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] PATH_INFO: `/v3/projects/f1be985d7217439ea29884d4aa108c02/users/d574061312164a879d7485b49c718003/roles/af0d03236dab45f3a150b6281f50bb80` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.376674 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1be985d7217439ea29884d4aa108c02, user_id=d574061312164a879d7485b49c718003, role_id=af0d03236dab45f3a150b6281f50bb80)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:14.380353 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.489106 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1454/2915] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 11:00:14 2022] PUT /identity/v3/projects/f1be985d7217439ea29884d4aa108c02/users/d574061312164a879d7485b49c718003/roles/af0d03236dab45f3a150b6281f50bb80 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:14.499960 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1105c-d0d4-43cf-a66a-971b2367017b None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.500255 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1105c-d0d4-43cf-a66a-971b2367017b None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.500599 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1105c-d0d4-43cf-a66a-971b2367017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.528045 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-f9f1105c-d0d4-43cf-a66a-971b2367017b None None] MFA Rules not processed for user `d574061312164a879d7485b49c718003`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 11:00:14.529236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-f9f1105c-d0d4-43cf-a66a-971b2367017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:14.592953 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1462/2916] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 11:00:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 11:00:14.597735 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:14.656964 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5669d-a6d2-49f1-89d5-0a13a12e5216 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:14.662136 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:14.662788 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.663070 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.663350 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.664027 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:14.665939 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.778923 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1455/2917] 10.222.0.35 () {60 vars in 1152 bytes} [Tue May 24 11:00:14 2022] POST /identity/v3/projects => generated 381 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 24 11:00:14.782783 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1105c-d0d4-43cf-a66a-971b2367017b None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:14.794194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1105c-d0d4-43cf-a66a-971b2367017b None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:14.849176 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:14.849835 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.850067 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.850342 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.851034 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:14.852972 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.965236 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1463/2918] 10.222.0.35 () {60 vars in 1143 bytes} [Tue May 24 11:00:14 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 24 11:00:14.969229 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:14.980376 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c6ee8-7d26-48a5-8ae2-473985242162 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:14.984772 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:14.985464 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:14.985791 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:14.986072 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:14.986825 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:14.988804 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:14.999439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1456/2919] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 11:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 24 11:00:15.003557 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:15.013966 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e814b22-1d5e-4b92-97b8-4d89f9e038c0 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:15.019036 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:15.019937 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:15.020218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:15.020490 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] PATH_INFO: `/v3/projects/6208f9cd061a40aeac3a978e4d0464db/users/4e95adbef98a4dbdb51ac161947ec841/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:15.035477 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6208f9cd061a40aeac3a978e4d0464db, user_id=4e95adbef98a4dbdb51ac161947ec841, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:15.038156 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:15.102396 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1464/2920] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 11:00:15 2022] PUT /identity/v3/projects/6208f9cd061a40aeac3a978e4d0464db/users/4e95adbef98a4dbdb51ac161947ec841/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:15.106305 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:15.116564 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91699f2a-a4e8-4fd0-9f59-c405af4b5eef admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:15.136445 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:15.172845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27573ea-bf0d-48f4-ae62-faed677432f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:15.173561 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27573ea-bf0d-48f4-ae62-faed677432f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:15.173847 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27573ea-bf0d-48f4-ae62-faed677432f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:15.174126 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27573ea-bf0d-48f4-ae62-faed677432f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:15.174948 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27573ea-bf0d-48f4-ae62-faed677432f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:15.176878 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27573ea-bf0d-48f4-ae62-faed677432f6 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:15.187341 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1457/2921] 10.222.0.35 () {58 vars in 1121 bytes} [Tue May 24 11:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 24 11:00:15.192051 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:15.197462 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:15.202732 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce529a2e-1e35-461d-bb02-6252411d5226 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:15.208138 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:15.209296 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:15.209577 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:15.215437 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] PATH_INFO: `/v3/projects/6208f9cd061a40aeac3a978e4d0464db/users/4e95adbef98a4dbdb51ac161947ec841/roles/e1e5746208514068acadbab89907c473` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:15.215437 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6208f9cd061a40aeac3a978e4d0464db, user_id=4e95adbef98a4dbdb51ac161947ec841, role_id=e1e5746208514068acadbab89907c473)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:15.215437 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:15.224970 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1465/2922] 10.222.0.35 () {60 vars in 1482 bytes} [Tue May 24 11:00:15 2022] PUT /identity/v3/projects/6208f9cd061a40aeac3a978e4d0464db/users/4e95adbef98a4dbdb51ac161947ec841/roles/e1e5746208514068acadbab89907c473 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:15.234942 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bb39d-8741-4efd-a8ab-9ad6a32e2241 None None] REQUEST_METHOD: `POST` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:15.235233 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bb39d-8741-4efd-a8ab-9ad6a32e2241 None None] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:15.235513 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50bb39d-8741-4efd-a8ab-9ad6a32e2241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:15.261885 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.auth.core [None req-b50bb39d-8741-4efd-a8ab-9ad6a32e2241 None None] MFA Rules not processed for user `4e95adbef98a4dbdb51ac161947ec841`. Rule list: `[]` (Enabled: `True`). {{(pid=80404) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 11:00:15.263845 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-b50bb39d-8741-4efd-a8ab-9ad6a32e2241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:15.331743 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1458/2923] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 11:00:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 11:00:15.342467 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:15.344143 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:15.400003 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61690572-96a7-4464-87a2-3750d2f6fdf7 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:15.403551 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52b0a-17f4-411e-8ea3-d9b5e708f1c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aaba1a8ba921489e96f9fa4acecffb74', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:15.404194 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52b0a-17f4-411e-8ea3-d9b5e708f1c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:15.404474 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52b0a-17f4-411e-8ea3-d9b5e708f1c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:15.404749 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52b0a-17f4-411e-8ea3-d9b5e708f1c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:15.413679 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e52b0a-17f4-411e-8ea3-d9b5e708f1c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:15.415111 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e52b0a-17f4-411e-8ea3-d9b5e708f1c0 service cinder] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:15.424304 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1466/2924] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 11:00:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 11:00:15.870130 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bb39d-8741-4efd-a8ab-9ad6a32e2241 None None] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:15.931955 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50bb39d-8741-4efd-a8ab-9ad6a32e2241 None None] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:15.936529 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:15.937427 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:15.937709 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:15.937981 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] PATH_INFO: `/v3/users/d574061312164a879d7485b49c718003` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:15.939159 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d574061312164a879d7485b49c718003)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:15.941795 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:16.093894 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd574061312164a879d7485b49c718003'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:16.094553 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd574061312164a879d7485b49c718003'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:16.095773 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:16.102019 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd574061312164a879d7485b49c718003'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:16.102652 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd574061312164a879d7485b49c718003'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:16.115050 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd574061312164a879d7485b49c718003'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:16.205621 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1459/2925] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 11:00:15 2022] DELETE /identity/v3/users/d574061312164a879d7485b49c718003 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:16.210991 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4918-7511-450b-8236-aa0647c81529 None None] REQUEST_METHOD: `POST` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:16.211449 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4918-7511-450b-8236-aa0647c81529 None None] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:16.211544 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f4918-7511-450b-8236-aa0647c81529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:16.228904 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.auth.core [None req-330f4918-7511-450b-8236-aa0647c81529 None None] MFA Rules not processed for user `db8edbaf5787451c9cd851df007d0f35`. Rule list: `[]` (Enabled: `True`). {{(pid=80405) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 24 11:00:16.230105 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-330f4918-7511-450b-8236-aa0647c81529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:16.287848 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1467/2926] 10.222.0.35 () {58 vars in 957 bytes} [Tue May 24 11:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 24 11:00:16.297439 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:16.298829 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:16.357038 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95f717-01d3-4bf3-8ff1-b97213e88a4a admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:16.360643 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9937f1088b304574935fcdf579e4de7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:16.361307 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:16.361595 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:16.361872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:16.371409 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:16.372842 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:16.382819 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1460/2927] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 11:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 24 11:00:16.788014 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f4918-7511-450b-8236-aa0647c81529 None None] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:16.790257 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-330f4918-7511-450b-8236-aa0647c81529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:16.804129 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f4918-7511-450b-8236-aa0647c81529 None None] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:16.809416 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:16.810465 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:16.810746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:16.811023 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] PATH_INFO: `/v3/projects/f1be985d7217439ea29884d4aa108c02` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:16.812192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1be985d7217439ea29884d4aa108c02)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:16.814173 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:16.917756 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1be985d7217439ea29884d4aa108c02', 'request_id': 'req-78426f30-658e-4961-ba17-7c0144079367'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.005192 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1be985d7217439ea29884d4aa108c02', 'request_id': 'req-78426f30-658e-4961-ba17-7c0144079367'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.011708 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1be985d7217439ea29884d4aa108c02', 'request_id': 'req-78426f30-658e-4961-ba17-7c0144079367'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.022772 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] The token cache is being invalidate because project f1be985d7217439ea29884d4aa108c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 11:00:17.023063 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.023698 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.025248 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1468/2928] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 11:00:16 2022] DELETE /identity/v3/projects/f1be985d7217439ea29884d4aa108c02 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:17.029477 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:17.040420 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24d489-2b8d-4e08-93bd-d0e03d071261 service neutron] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:17.096230 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:17.097978 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:17.098261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:17.098537 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] PATH_INFO: `/v3/users/4e95adbef98a4dbdb51ac161947ec841` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:17.099722 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e95adbef98a4dbdb51ac161947ec841)` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:17.101682 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:17.285984 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e95adbef98a4dbdb51ac161947ec841'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.286648 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e95adbef98a4dbdb51ac161947ec841'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.293675 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e95adbef98a4dbdb51ac161947ec841'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.294241 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e95adbef98a4dbdb51ac161947ec841'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.304047 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:17.306261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.notifications [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e95adbef98a4dbdb51ac161947ec841'} {{(pid=80404) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:17.387785 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1461/2929] 10.222.0.35 () {58 vars in 1223 bytes} [Tue May 24 11:00:17 2022] DELETE /identity/v3/users/4e95adbef98a4dbdb51ac161947ec841 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 24 11:00:17.764122 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Authenticating user token {{(pid=80405) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:00:17.765684 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.fernet_utils [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80405) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:00:17.778744 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78426f30-658e-4961-ba17-7c0144079367 admin admin] Validating token access rules against request {{(pid=80405) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:00:17.792553 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:17.838730 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db8edbaf5787451c9cd851df007d0f35', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd39d97f2faca4faba12c96da090b5017', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80405) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:00:17.839530 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:00:17.839817 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:00:17.840095 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] PATH_INFO: `/v3/projects/6208f9cd061a40aeac3a978e4d0464db` {{(pid=80405) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:00:17.841402 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6208f9cd061a40aeac3a978e4d0464db)` {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:00:17.843218 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] RBAC: Authorization granted {{(pid=80405) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:00:17.980338 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6208f9cd061a40aeac3a978e4d0464db', 'request_id': 'req-d18db66a-40c5-494b-9b6f-be805bab54ff'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:18.048298 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6208f9cd061a40aeac3a978e4d0464db', 'request_id': 'req-d18db66a-40c5-494b-9b6f-be805bab54ff'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:18.055746 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6208f9cd061a40aeac3a978e4d0464db', 'request_id': 'req-d18db66a-40c5-494b-9b6f-be805bab54ff'} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:18.067041 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] The token cache is being invalidate because project 6208f9cd061a40aeac3a978e4d0464db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80405) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 24 11:00:18.067433 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:18.068071 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG keystone.notifications [None req-d18db66a-40c5-494b-9b6f-be805bab54ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80405) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 24 11:00:18.069653 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: [pid: 80405|app: 0|req: 1469/2930] 10.222.0.35 () {58 vars in 1232 bytes} [Tue May 24 11:00:17 2022] DELETE /identity/v3/projects/6208f9cd061a40aeac3a978e4d0464db => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 24 11:00:27.433583 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:DELETE=2,keystone:SELECT=3,keystone:INSERT=1 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:00:28.063331 dv-cinder-843118-2-iscsi devstack@keystone.service[80405]: DEBUG dbcounter [-] [80405] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80405) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 24 11:02:11.087261 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Authenticating user token {{(pid=80404) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 24 11:02:11.088641 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:02:11.170809 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c97a0c-d9e2-429c-a5a4-57d1b44c6048 admin admin] Validating token access rules against request {{(pid=80404) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 24 11:02:11.230509 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c3165-ffe0-480e-924e-a5fa8f820787 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4989f980a8804f18b7ea2fec0ecad8a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bfce8a80cde4eadbdcc4c830e8c06fc', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80404) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 24 11:02:11.231160 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c3165-ffe0-480e-924e-a5fa8f820787 service placement] REQUEST_METHOD: `GET` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 24 11:02:11.231449 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c3165-ffe0-480e-924e-a5fa8f820787 service placement] SCRIPT_NAME: `/identity` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 24 11:02:11.231732 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c3165-ffe0-480e-924e-a5fa8f820787 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80404) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 24 11:02:11.233543 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.fernet_utils [None req-907c3165-ffe0-480e-924e-a5fa8f820787 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80404) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 24 11:02:11.244668 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c3165-ffe0-480e-924e-a5fa8f820787 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 24 11:02:11.246987 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c3165-ffe0-480e-924e-a5fa8f820787 service placement] RBAC: Authorization granted {{(pid=80404) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 24 11:02:11.256872 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: [pid: 80404|app: 0|req: 1462/2931] 10.222.0.35 () {60 vars in 1365 bytes} [Tue May 24 11:02:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 24 11:02:21.252612 dv-cinder-843118-2-iscsi devstack@keystone.service[80404]: DEBUG dbcounter [-] [80404] Writing DB stats keystone:SELECT=63 {{(pid=80404) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}